Definition
Plain language
A benchmark of ten genuinely hard math problems contributed by working research mathematicians.
As stated in the literature
A research-math benchmark of ten problems contributed by mathematicians including Dan Spielman, Martin Hairer, Andrew Blumberg, and Shmuel Weinberger, used to evaluate RMA, GPT-5.2R, and Aletheia.
Why it matters: Benchmarks contributed by working researchers test whether AI can engage with real mathematics rather than just polished competition problems.
For example, one First Proof problem might come straight from an open question Martin Hairer has been thinking about, rather than being adapted from a textbook.
Heard on the show
“There's a new benchmark called First Proof.”Episode 076 — Same Model, Organized Differently: How an Agent Architecture Beat Frontier Systems at Research Math