Definition
Plain language
The sudden collapse in accuracy when a state-space model's running summary can no longer hold a needed fact.
As stated in the literature
The empirical phenomenon in state-space models where recall accuracy abruptly drops to chance once the gap between a stored binding and its retrieval exceeds the recurrent state's effective horizon.
Why it matters: It's a sharp practical limit on long-context recall for SSMs that hybrid architectures and memory tricks are explicitly designed to push back.
For example, a state-space model accurately recalls a name introduced 5,000 tokens ago but completely loses it once the gap reaches 12,000 tokens, with accuracy collapsing to chance.
Heard on the show
“The authors have this phrase — the memory cliff — which I think is exactly the right metaphor.”Episode 033 — Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval