Definition
Plain language
A multiple-choice algebraic reasoning benchmark with word-puzzle style problems.
As stated in the literature
AQuA-RAT, a benchmark of multiple-choice algebraic word problems with rationales, used in arithmetic and reasoning evaluations.
Why it matters: It's a long-standing benchmark for whether models can both reason about word problems and produce explicit rationales, not just guess the answer.
For example, an AQuA problem might describe a train scenario with several given quantities and ask you to pick the correct equation among five choices.
Heard on the show
“They do touch on it, with the AQuA case.”Episode 013 — Why Search Keeps Rediscovering the Same Workflow, and What That Means