Definition
Plain language
When a model's internal sense of 'where I am' in a long step-by-step task drifts away from reality, with errors snowballing as it goes.
As stated in the literature
The architectural failure mode in which a transformer's fixed-capacity attention cannot keep enough of its own history in focus, so tracked state diverges from ground truth and per-step error grows with depth, producing super-exponential accuracy collapse.
Also called: decoherence
Why it matters: It matters because it pins long-task failure on a fixed architectural limit rather than something more training or prompting could fix.
For example, asked to track a long sequence of moves, a model's internal sense of where it stands slowly drifts off course, and small errors pile up faster and faster until the answer collapses.
Heard on the show
“That's the failure mode the authors name State-Space Decoherence — your internal picture of "where I am" decoheres from reality.”Episode 108 — The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks