What is the difference between offline and online evaluation for AI systems?

Offline evaluation runs before deployment on a fixed dataset — you score model or prompt versions against golden examples in CI to catch regressions and compare candidates. It's fast, reproducible, and gates releases, but it only measures what your dataset covers and can't capture real user behaviour. Online evaluation happens in production on live traffic: A/B tests, user signals (thumbs up/down, edits, task completion, retention), and sampling real outputs for human or automated review. It reflects reality but is slower, noisier, and riskier. You need both — offline to ship with confidence and prevent obvious regressions, online to learn what actually works for users and to catch drift the offline set never anticipated.