AI Benchmarks Explained: How We Measure Model Performance

Every week brings a new AI model with a chart claiming it beats GPT-5.6, Claude, or Gemini on some benchmark. UK businesses trying to pick a model for real work are stuck reading these charts with no idea what they actually measure. When I looked into how these numbers get made, the gap between the marketing slide and the underlying test was bigger than expected. Nine points on a leaderboard sound decisive. In practice they can mean almost nothing for the task sat in front of you.

What Is an AI Benchmark, Really?

A benchmark is just a fixed set of questions or tasks with known correct answers. A model runs through the set, its answers get scored, and the percentage correct becomes the headline number. Simple in theory.

In practice, benchmarks range from multiple-choice trivia to full coding projects. MMLU, one of the most cited, contains 15,908 questions across 57 subjects, from law to virology. A model scoring 88% on MMLU sounds impressive. It says almost nothing about whether that model can draft a UK employment contract without hallucinating a clause.

There are now over 200 named benchmarks in active use across the industry, covering everything from basic arithmetic to long-form creative writing. Most companies quote three or four in a press release and quietly skip the ones where their model underperforms.

MMLU and the Multiple-Choice Trap

MMLU tests recognition, not reasoning. A model can score well by pattern-matching against training data that included similar questions. That’s not the same skill as solving a brand-new problem.

Researchers at Stanford flagged that several MMLU questions contain outright errors — wrong answers marked as correct. Models trained to match the “correct” answer sometimes learn the mistake, not the truth. Falls apart fast once you dig in.

Newer benchmarks like MMLU-Pro were built specifically to fix this, adding harder distractors and cutting the guessable options from four to ten. Scores dropped across every major model when the test got harder — a sign the original numbers were inflated. One lab’s flagship model fell from 91% on the original MMLU to 68% on MMLU-Pro, a 23-point gap that never made it onto any marketing page.

Coding Benchmarks: SWE-bench and the Real-World Test

SWE-bench is different. It takes real GitHub issues from real open-source projects and asks a model to fix them, then checks if the existing test suite passes. No multiple choice. No shortcuts.

Top models now clear over 70% of SWE-bench Verified tasks, up from under 20% two years ago. That jump is real progress, not just better test-taking. UK software teams use this benchmark more than any other when picking a coding assistant, because it maps to actual work.

Still, SWE-bench draws from Python repositories almost exclusively. A model’s score here tells you little about how it handles COBOL, or a bespoke internal system built in 2009. Worth remembering before you trust the leaderboard blindly.

A newer variant, SWE-bench Multimodal, adds tasks involving screenshots and visual bugs — the kind of ticket a front-end developer actually gets. Scores there sit roughly 15 points lower across the board, a more honest picture of current limits.

Why Benchmarks Get Gamed

Training data contamination is the industry’s open secret. If a benchmark’s questions leaked onto the public internet, and a model trained on internet text, the model may have memorised the answers rather than reasoned them out.

Some labs now hold back a portion of benchmark questions, releasing only enough to verify results without exposing the full set. Others build private evaluation sets that never touch the internet at all. Neither approach is universal yet.

UK investors keep asking about this because a chatbot that scores 95% on a public leaderboard can still fumble a simple customer email. The benchmark and the job aren’t always the same task, dressed differently.

There’s also a subtler version of gaming: fine-tuning a model specifically on benchmark-adjacent data right before a release, boosting the score without improving general capability. Researchers call this “benchmark overfitting,” and it’s nearly impossible to detect from outside a lab.

Human Preference Tests: The Chatbot Arena Approach

Chatbot Arena takes a different route entirely. Two models answer the same prompt anonymously, a human picks the better response, and an Elo rating system — borrowed from chess — ranks the models over millions of votes.

This captures something benchmarks miss: does the answer actually feel useful to a person? A model can ace a maths test and still write clunky, over-hedged prose that nobody wants to read.

The catch is that human raters tend to prefer longer, more confident-sounding answers, even when they’re wrong. Style beats substance sometimes. It’s a genuine flaw, and one the Arena’s own maintainers have acknowledged publicly.

Over four million votes have been cast on the platform since it launched, making it one of the largest live evaluations of AI models anywhere. That scale is a strength. It doesn’t fix the bias problem, though.