Definition
Plain language
The number of exact step-by-step moves a model can reliably track before its internal picture of where it is falls apart.
As stated in the literature
The reasoning depth past which a decoder-only transformer can no longer reliably track exact state, set by architectural attention capacity (head count and width) rather than context length, training, or inference budget; failures collapse super-exponentially beyond it.
Why it matters: It matters because it sets a hard ceiling on how many exact sequential steps a model can truly track, no matter how much you let it think or how long its context.
For example, a model might flawlessly follow a recipe of exact moves for the first dozen steps but then lose track of its position once the chain stretches past its deterministic horizon.
Heard on the show
“… And that "deterministic horizon" in the title is the thing to hold onto — because the paper's claim is that there's a specific …”Episode 108 — The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks