Definition
Plain language
A measure of how spread out a model's focus is at each step — high means its attention is smeared thin across many positions.
As stated in the literature
The entropy of a transformer's attention weight distribution; in deterministic-horizon work it grows roughly linearly with reasoning depth and correlates negatively with accuracy, strongest in late layers, evidencing attention dilution.
Why it matters: It matters because rising attention entropy is a measurable warning sign that a model's reasoning is losing precision as the task gets deeper.
For example, when a model is asked to track a long chain of steps, its attention entropy climbs as its focus gets smeared thinly across too many earlier positions instead of homing in on what matters.
Heard on the show
“They pull out attention entropy step by step.”Episode 108 — The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks