Definition
Plain language
A reasoning benchmark of small visual puzzles designed to be easy for humans and hard for AI.
As stated in the literature
The Abstraction and Reasoning Corpus, an evaluation suite of grid-based abstract reasoning tasks used to probe general intelligence in AI systems.
Also called: ARC-AGI-2, ARC, ARC-Challenge
Why it matters: It's one of the few benchmarks designed specifically to resist the kind of pattern-matching that lets models ace tests by memorization.
For example, a typical ARC-AGI puzzle shows a few before-and-after grid pairs and asks you to apply the same transformation to a new grid.
Heard on the show
“That's the founding idea behind ARC: measure how fast you pick up a brand-new skill, not how much you've already absorbed.”Episode 191 — How One Researcher Beat GPT-5.2 and Gemini 3 by Judging Their Answers, Not Improving Them