How to Read an AI Benchmark Score, and When to Distrust It

What the number literally means

Most language model benchmarks are, physically, a file of questions. MMLU is 14,042 four-option multiple-choice questions across 57 subjects. GSM8K's test split is 1,319 grade-school word problems. HumanEval is 164 Python functions, each checked by unit tests the model never sees. A table saying a model scored 88 claims exactly this: on that file, under that grading rule, that share of items came out right. The rest is inference.

The inference is often fair, and it weakens with distance. A HumanEval problem is a few lines long and self-contained; the docstring describes it, but the hidden tests decide whether you passed, and they routinely check edge cases the docstring never mentions. Real work rarely has that shape: tasks arrive underspecified, inside a codebase with local conventions, with the acceptance criteria in someone's head. That gap is not a defect in the benchmark; it is the definition of one, a gradeable proxy chosen because grading real work is expensive.

Grading choices move numbers as much as capability does. Some suites report the best of ten attempts, others a single shot. Prompt wording, the number of worked examples, even the order of the options can shift a result several points on an unchanged model, so two honest publications can report different numbers for the same model on the same test.

When the model has already seen the test

Public benchmarks live in public: code repositories and dataset hubs, quoted in tutorials, pasted into forum answers, reproduced in papers and blog posts, translated. Nobody has to cheat for a test set to end up in training data; that is the default outcome of scraping the same internet the benchmark was published to.

Filtering is harder than it sounds. Canary strings, the unique markers embedded in a dataset so trainers can exclude it, work only while copies keep them, and a reformatted table in a blog post does not. Hash-based deduplication misses paraphrases and translations carrying the same answer. A memorised test set gives a high number with no generalisation behind it.

Contamination is rarely provable from outside, since training data is usually undisclosed, but you can watch for its signature. In 2024 a team at Scale AI commissioned GSM1k: 1,000 new grade-school problems matched to GSM8K in style and difficulty (arXiv 2405.00332). Several families scored clearly lower on the fresh set, the largest gaps among the Phi and Mistral models, while Gemini, GPT and Claude showed little difference. That is evidence of contamination rather than proof of it. Still, a score that survives a rewritten or private test is worth more than the same score on a set that has been online for years.

Who ran the evaluation

Numbers from the lab that built the model are not lies by default. It has the best access, the most compute, and every reason to measure carefully before shipping. But it also chooses the prompt, the number of examples, the decoding settings, which subset to report and when to stop running experiments. Nobody has to falsify anything for a published figure to be the friendliest of several honest measurements.

Independent evaluation removes that pressure and adds others. An outside harness may use defaults the model was not built for, omit the system prompt and tools the product ships with, or quietly sample fewer items because of rate limits. A careful in-house run can beat a sloppy external one: the source of a number is a weak signal, the method a strong one.

The test is not who published a figure but whether it comes with enough to reproduce it: which model version, on what date, how many items, how many examples in the prompt, what decoding settings, what grading rule. A figure with that attached is a claim you can check. One without it is a marketing asset with a decimal point.

Small gaps are mostly sampling error

Benchmarks are samples, so scores carry error even when everything else is held fixed. GPQA Diamond has 198 questions; at around 50% accuracy the standard error on a single score is about 3.5 points. HumanEval's 164 problems mean scores land only on multiples of 0.61, so a reported 88.4 is 145 problems out of 164. Comparing two models is a different calculation. A three-point gap on GPQA Diamond is six items; whether it is real depends on how much the two models disagree elsewhere, because the honest test looks only at the items where they differ, and six is rarely enough to clear it.

Sampling error is only the floor. Above it sit run-to-run variance whenever temperature is above zero and differences between evaluation harnesses. When two models are within a few points, decide on what can be measured without ambiguity: price, latency, context length, and whether the service is available where you are at all.

Why a good score may not help you

The failures that hurt in production are largely not the ones benchmarks measure: the model that breaks your JSON schema, drops a constraint by the twelfth turn, invents a plausible citation, or answers correctly but too slowly for the interaction you are building. None of that is graded in the table you are reading.

The evaluation that predicts your outcome is the one you build: thirty to fifty real cases from your own traffic, with answers you consider correct and a grading rule you would defend, re-run whenever you change model or prompt. A set that size will not resolve a five-point difference between two models, because the arithmetic above still applies, but ranking is not its job. It is there to catch the failures that matter to you and to show when a change has broken something that worked. Evaluate the whole system, prompt and retrieval and tools and output format together: the same model scores differently depending on what you wrap around it, so a system comparison and a model comparison are not the same experiment.

What an Arabic evaluation has to do differently

A translated benchmark imports the assumptions of its source. Questions about American case law, or the history taught in American high schools, rendered into Arabic, test translation and trivia rather than what an Arabic-speaking user needs to know. Machine translation adds a second problem: an error can leave an item with no correct answer, or change which answer is correct, and the model is then graded on the translator's mistake. Human translation fixes the wording, not the cultural frame.

Arabic brings failure modes English suites never touch. A model can be fluent in formal Arabic and lose the thread in Iraqi or Gulf dialect. Orthographic variation, the forms of alef, where the hamza sits, ta marbuta against ha, optional diacritics, is punished by a string-matching grader that is really measuring spelling convention. Arabic also costs more tokens per word than English in current tokenisers, so the same quality costs more and the context window fills sooner; an evaluation reporting accuracy alone has left out the part of the bill your users pay. A serious Arabic suite is written in Arabic, by Arabic speakers, about what people here actually need, and states its normalisation rules before it states a number.