Definition
Plain language
In sequence modeling, the link between a key and its associated value that the model has to recall later.
As stated in the literature
A key-value association stored implicitly in a recurrent or attention-based model, whose retrievability degrades with distance in state-space models.
Also called: bindings
Why it matters: Binding is exactly what state-space models tend to lose at long range, which is why associative-recall tests are such a sharp differentiator from transformers.
For example, if a model reads 'Alice is 32 and Bob is 47' and is later asked Alice's age, recalling 32 requires keeping the binding between 'Alice' and '32' intact.
Heard on the show
“So where does that leave the fix, because binding the label sounds easy.”Episode 238 — How a Cheap Model Reads the Flagship's Secret Reasoning Aloud