Definition
Plain language
A way of generating text that only picks from the most likely next-word candidates, ignoring the long tail.
As stated in the literature
A decoding method that samples from the smallest set of tokens whose cumulative probability exceeds a threshold p, also called top-p sampling.
Why it matters: It keeps generations diverse without dipping into the long tail of bizarre tokens that pure temperature sampling would occasionally produce.
For example, with p=0.9 the model only samples from the smallest set of next-word candidates whose probabilities sum to 90 percent.
Heard on the show
“They use an adaptive threshold — same idea as nucleus sampling, but applied to prompts ranked by variance instead of tokens ranked by probability.”Episode 010 — When Reward Climbs But Reasoning Goes Generic: Diagnosing Template Collapse in Agentic RL