Definition
Plain language
A speed trick where a model only pays attention to a chosen subset of earlier words instead of all of them.
As stated in the literature
A class of attention approximations that compute scores against a selected subset of tokens to reduce cost; reframed in recent work as a halfspace range-searching problem with exactness guarantees rather than approximate nearest-neighbor retrieval.
Why it matters: It cuts the steep cost of attention on long inputs, and framing it as an exact search problem can preserve accuracy while saving compute.
For example, when processing a long document, the model only attends to a chosen handful of earlier words rather than every single one.