GPQA Diamond went from GPT-4’s 38.8% in late 2023 to a three-way photo finish around 94% by mid-2026 — and independent analysis suggests a meaningful share of what remains unsolved is flawed questions, not unsolved science. If a vendor is still leading a launch post with a GPQA number, that tells you more about their marketing calendar than their model. Here is what the benchmark actually measures, where the scores stand today, and why the last few points no longer separate models in any meaningful way.
What GPQA actually measures
GPQA (Graduate-Level Google-Proof Q&A) was introduced by Rein et al. in November 2023. It is a set of 448 four-way multiple-choice questions in biology, physics, and chemistry, written by people holding or pursuing PhDs in those fields. The design goal was in the name: questions had to be Google-proof. Skilled non-expert validators — people with PhDs in other fields, given over 30 minutes of unrestricted web access per question — reached only 34% accuracy. Domain experts scored 65%, rising to about 74% once questions the experts themselves flagged as flawed were discounted.
That last number matters more than it first appears. If expert consensus only covers ~74% of the answer key, the benchmark’s own authors effectively documented a ceiling of uncontroversial correctness well below 100%. Keep that in mind when reading 94% scores.
GPQA Diamond is the subset almost everyone actually reports: 198 questions where both independent expert annotators answered correctly and the majority of non-expert validators answered incorrectly. It is the hardest, cleanest slice — and at 198 questions, a single question is worth roughly half a percentage point. Random chance is 25%.
The scoreboard, July 2026
Frontier scores are clustered so tightly that the trackers no longer agree on who is on top — and the disagreement is itself the finding.
- Artificial Analysis (independent re-runs, July 2026) puts GPT-5.6 Sol (max) and Gemini 3.1 Pro Preview at the top at ~94.1%, with GPT-5.5 (xhigh) at 93.5%.
- LLM-Stats (July 2026) shows Gemini 3.1 Pro at 94.1%, GPT-5.5 at 94.0%, and Claude Opus 4.8 at 93.6%.
- Other snapshots the same month report Gemini 3.1 Pro anywhere from 94.3% to 95.45% depending on prompting configuration and which release is tested.
Read that spread carefully: the gap between first and third place across trackers is which questions land on either side of a coin flip. On a 198-question test, a “two-point lead” is four questions. Treating these rankings as a capability ordering is statistical malpractice.
How fast it saturated
The trajectory from sub-expert to expert-surpassing took roughly two years, per Epoch AI’s benchmark tracking and the original paper:
| Date | Model | GPQA Diamond | Source |
|---|---|---|---|
| Nov 2023 | GPT-4 | 38.8% | Rein et al. |
| Nov 2023 | PhD expert baseline | 65% (74% adjusted) | Rein et al. |
| Sep 2024 | OpenAI o1 | 77–78% | Epoch AI |
| Early 2025 | OpenAI o3 / Gemini 2.5 Pro | 83.3% / ~86.4% | Epoch AI |
| Mid 2025 | Frontier cluster | ~91–92.4% | LLM-Stats |
| Jul 2026 | Gemini 3.1 Pro / GPT-5.5 / Claude Opus 4.8 | 93.6–94.1% (tracker-dependent) | Artificial Analysis, LLM-Stats |
The first model to cleanly beat the human PhD-expert baseline was o1 in September 2024. Twenty-two months later, the frontier sits more than 20 points above that baseline, and the curve has visibly flattened into an asymptote.
What’s actually left in the last 6%
When models approach 100% on any benchmark, the correct question is: what if the remainder is flawed? Epoch AI’s Greg Burnham ran exactly this analysis in “GPQA Diamond: What’s left?”, and the findings are more nuanced than either the “it’s solved” or “it’s broken” camp would like.
Frontier models collectively score over 50% on 158 of the 198 questions. Their wrong answers cluster heavily on the remaining 40 — and most of those 40 are organic chemistry, a domain with its own notational quirks. Burnham’s manual review found roughly 2–3 questions that are likely invalid outright (bad answer key or unsolvable), putting the benchmark at ~90–95% valid. That is better hygiene than critics assumed, but it still means the frontier is now fighting over territory where question quality and model capability are hard to distinguish. The questions models miss on a near-solved benchmark are disproportionately the ambiguous ones, the mislabeled ones, and the ones with contested answers.
There is also a subtler failure mode Burnham documented: on some physics questions, models briefly consider the correct solution path in their chain of thought, discard it, and commit to a plug-in-numbers approach. Asked directly for the intermediate steps, they produce correct values. The capability is present; the search over solution strategies fails. That is a real, interesting limitation — the kind we track in our Limitations Ledger — but it is invisible in a single top-line percentage.
Why it saturated so fast
Two mechanisms compressed the timeline. The first was the shift to test-time reasoning: o1’s September 2024 jump from GPT-4-class ~40% scores to 77–78% came almost entirely from spending more compute at inference, not from new science knowledge. Once every lab shipped a reasoning mode, the expert baseline stopped being a ceiling and became a mile marker. The second is subtler: multiple-choice format compounds with reasoning. A model that can eliminate two of four options and reason carefully about the remaining pair converts partial knowledge into full credit at a rate no free-response format would allow. GPQA was designed to resist web search, not to resist extended chains of thought — a threat model that barely existed when the questions were written in 2023.
One reporting trap worth flagging: “GPQA” without a qualifier is ambiguous. Vendors overwhelmingly report the 198-question Diamond subset, but some papers and older model cards cite the 448-question main set, where scores run lower. When two numbers for the same model differ by several points, check which set was used before assuming a regression.
How to read a GPQA score in 2026
GPQA Diamond still has a legitimate use: it is a fast, cheap sanity check that a model has frontier-class scientific reasoning, and vendor self-reported scores have tracked independent re-runs reasonably well. What it can no longer do is rank frontier models against each other. Differences under ~2 points are noise on a 198-item test; differences across trackers of a full point for the same model are routine, driven by prompting, sampling, and which checkpoint was tested.
If you are building an evaluation stack, treat GPQA the way we describe in our guide to evaluating LLMs: as one saturated signal among many, useful as a floor check, useless as a tiebreaker. The discriminating benchmarks have moved on — harder science sets, agentic tasks, and rolling-refresh designs that resist contamination. GPQA’s own authors flagged ~26% of questions as lacking full expert consensus in 2023; at 94%, models are now operating inside that error bar.
FAQ
Is GPQA Diamond contaminated?
Partially, and unavoidably: the dataset has been public since November 2023 and benchmark items have been found in training corpora. But Epoch AI’s question-level analysis suggests the remaining misses look more like genuine reasoning failures and flawed items than memorization gaps, so contamination is not the dominant story — saturation is.
What does a 94% GPQA score actually mean?
It means the model answers graduate-level, search-resistant science multiple-choice questions well above the 65–74% documented PhD-expert baseline. It does not mean the model does science: GPQA is four-option multiple choice, which rewards elimination strategies no working scientist gets to use.
Which model is best on GPQA right now?
As of July 2026, Gemini 3.1 Pro, GPT-5.5/5.6, and Claude Opus 4.8 all sit between roughly 93.5% and 94.1% depending on the tracker — inside each other’s error bars. Any claim of a decisive leader on this benchmark is a rounding artifact.
Last updated July 16, 2026. This page is refreshed as benchmarks and scores move.