AI Benchmarks: How Do We Know Which Model Is Better?
Learn how models are compared with benchmarks, where those comparisons mislead, and how to test for your own needs.
Every new model arrives with a table of impressive scores. How are those numbers made, and how much should you trust them?
A benchmark is a standard exam for models: a fixed set of questions with known answers, scored the same way for everyone. Some test knowledge across many subjects (MMLU, multiple choice), some school maths word problems (GSM8K), some real programming tasks checked by automated tests (SWE-bench). Others rank models by human taste: in arenas like LMArena, people compare two anonymous answers side by side and vote for the better one, and the votes become a chess-style rating.
Benchmarks are genuinely useful, but they have three classic weak spots:
- Contamination. Test questions published online can end up in the training data, so the model may have seen the answers. Scores on a fresh, unseen version are often lower.
- Saturation. When every model scores 95%, the test can no longer tell them apart, and new, harder benchmarks have to be invented.
- Measuring the wrong thing. A multiple-choice quiz isn’t your job. Arena votes can favour longer, friendlier answers over correct ones.
So treat leaderboards as a shortlist, not a verdict. The test that matters most is the one only you can run: collect 20 to 50 real examples of your task, with what a good answer looks like, and try your shortlisted models on them. It often reveals that a cheaper model is just as good, or that the leaderboard champion stumbles on exactly the thing you need.
Key takeaways
- Benchmarks are standard exams for models: knowledge, maths, coding, or human preference votes.
- Contamination (seen the answers) and saturation (everyone scores high) make scores less meaningful.
- A benchmark measures one thing, which may not be your thing.
- Use leaderboards to shortlist, then test on 20–50 of your own real examples.
Check your understanding
Go deeper (more technical)
MMLU (2020) covers 57 subjects in multiple-choice form; GSM8K (2021) has about 8,500 grade-school maths problems; SWE-bench (2023) uses real GitHub issues checked by each project’s tests; LMArena (formerly Chatbot Arena) ranks models from millions of crowd votes using an Elo-style system. Harder successors keep appearing as old tests saturate.
Building your own test set is called evaluation (“evals”). Teams keep it alongside their app and rerun it whenever they change the prompt or the model, so they notice when something gets worse. Track 9 returns to how to judge whether an answer is good.
The simulation’s models and scores are invented.