Definition
Plain language
Keeping only the highest-scoring few items and ignoring the rest.
As stated in the literature
A selection rule that retains the k entries with the largest scores, used both in attention sparsification and in decoding strategies.
Also called: top-k retrieval
Why it matters: It's a basic knob across generation and retrieval that trades diversity for quality, and the right k can sharply affect output behavior.
For example, top-k sampling with k=50 means the model only considers the 50 most likely next tokens and ignores the rest of the vocabulary.
Heard on the show
“They suggest the obvious next step — concept-aware decoding, where you cluster the top-k tokens by semantic equivalence before taking the argmax.”Episode 070 — When Models Know the Answer But Say the Wrong Thing Anyway