Definition
Plain language
A benchmark of multi-step arithmetic word problems used to test math reasoning.
As stated in the literature
A small dataset of multi-step arithmetic word problems, commonly used as an out-of-distribution check on math-reasoning agent workflows.
Why it matters: It's a clean, well-studied check on whether a reasoning method actually helps on chained arithmetic, useful as an out-of-domain probe.
For example, a problem might describe a shopkeeper buying apples, selling some, then restocking, and ask how many remain.
Heard on the show
“If you actually look at what AFlow finds for math — GSM8K, MATH, MultiArith — they all converge to roughly the same shape.”Episode 013 — Why Search Keeps Rediscovering the Same Workflow, and What That Means