Definition
Plain language
The part that picks which word the model actually says next, out of all the options it scored.
As stated in the literature
The decoding component that converts logits into a chosen token — temperature scaling, top-k/top-p truncation, repetition penalties — with per-engine differences in ordering and defaults.
Also called: samplers, sampling
Why it matters: It controls how predictable or creative the output feels, and its settings can turn the exact same model from a dependable summariser into an erratic one.
For example, if the model rates "cat" as most likely and "dog" as a close second, the sampler is what decides whether the sentence ends up saying cat or dog.
Heard on the show
“RL was sampling efficiency in a fancy outfit.”Episode 011 — When RL Actually Teaches Agents Something New, And When It Doesn't