Definition
Plain language
A standard set of competition math problems used to test reasoning models.
As stated in the literature
A 500-problem subset of the MATH benchmark, drawn from competition mathematics across algebra, geometry, and number theory.
Also called: MATH
Why it matters: It's one of the standard benchmarks reasoning-focused models report on, so it shapes which capabilities researchers actively chase.
For example, a MATH-500 problem might ask for the smallest positive integer satisfying a particular modular equation, requiring multi-step algebraic reasoning.
Heard on the show
“Category A is pure math, MATH-500, no tools.”Episode 011 — When RL Actually Teaches Agents Something New, And When It Doesn't