Definition
Plain language
A small set of extremely hard graduate-level science questions used to test AI reasoning.
As stated in the literature
The hardest subset of GPQA, a benchmark of expert-validated, Google-proof graduate-level science questions in physics, chemistry, and biology.
Also called: GPQA, G-P-Q-A-Diamond
Why it matters: As the hardest subset, it's where capability gaps between frontier models still show clear differences, making it a useful current discriminator.
For example, a GPQA-Diamond question might ask the experimental technique needed to distinguish two reaction mechanisms in organometallic chemistry — answerable by an expert but resistant to web search.
Heard on the show
“GPQA-Diamond: both Fugu variants hit 95.”Episode 166 — A Router That Beats the Frontier Models It Calls