eval

Evaluation is the broader practice of measuring how good a model actually is, and a benchmark — a fixed test set with a fixed scoring rule, covered under benchmark — is only one specific tool inside that broader practice. Evaluation also includes things no static benchmark can capture: live human preference judgments between two model outputs, A/B testing a model change against real user behavior in production, and red-teaming a model against adversarial inputs nobody wrote a labeled test set for. Treating "evaluation" and "benchmark score" as the same thing is the single most common mistake in how model quality gets reported, because a model can top every standard benchmark and still fail badly on the actual distribution of inputs it meets in production.

Human evaluation exists specifically for the judgments a fixed scoring rule can't make, and it comes with its own well-documented failure modes. Asking human raters which of two model responses is better captures nuances — tone, helpfulness, actual usefulness — that an automated metric like exact-match or BLEU score was never designed to measure, but human judgment is also expensive to collect at scale, inconsistent between raters, and vulnerable to superficial cues like response length or formatting swaying a rater's preference independent of actual quality — which is exactly why large-scale human evaluation platforms publish inter-rater agreement statistics alongside their results, rather than treating a single rater's judgment as ground truth.

Production A/B testing is the evaluation method that actually measures what a benchmark can only approximate: real user behavior. Rolling out a model change to a fraction of real traffic and measuring the actual downstream metric a business cares about — task completion, user retention, conversion — sidesteps the gap between benchmark performance and real-world performance entirely, because it is measuring the real thing rather than a proxy for it; the cost is that it requires a live product with real users already in motion, which makes it the slowest and most operationally demanding evaluation method, reserved for decisions big enough to justify the wait.

The core discipline across every evaluation method is the same leakage rule covered under cross-validation and validation-set: whatever data gets used to judge a model must never have influenced how that model was built. A benchmark whose test set leaked into pretraining data, a human-eval study where raters saw the training examples, or an A/B test contaminated by users who were also in the training data all commit the identical mistake in different clothing — using the answer key to grade the test — which is why evaluation methodology, not just evaluation results, is what a serious model comparison actually has to defend.

Evaluation is the question; a benchmark score is one specific, narrow answer to it — appropriate when the task genuinely has a fixed right answer and misleading whenever it's used as a stand-in for a broader claim about real-world quality that it was never actually designed to measure.