Definition
Plain language
A benchmark of logic-grid puzzles where each clue narrows down who lives where, who owns what, and so on.
As stated in the literature
A benchmark suite of multi-clue logic-grid puzzles used to test whether agentic LLM systems can offload constraint reasoning to formal solvers and recover the correct assignments.
Why it matters: It tests whether agents can recognize when a problem should be handed to a formal solver instead of solved by chain-of-thought guessing.
For example, a ZebraLogic puzzle might give clues like 'the doctor lives next to the cat owner' and ask which person owns which pet.
Heard on the show
“ZebraLogic, which is logic puzzles.”Episode 140 — When a Reasoning Model Says "Let Me Double-Check" After It's Already Decided