Definition
Plain language
The spread of answers a model tends to produce when you ask it the same thing many times.
As stated in the literature
The output distribution induced by a model and decoding settings; repeated sampling cannot recover set members that lie outside its support.
Why it matters: If an answer never appears in the model's output, no amount of retrying will find it, so repeated sampling fixes flakiness but not blind spots.
For example, asking the model for European capitals twenty times might surface Paris and Berlin every time and Valletta never, so Valletta is simply outside what it produces.
Heard on the show
“The dominant model has been "run many samples and aggregate" — and the implicit theory is that diversity in the sampling distribution will surface the right answer somewhere in the cloud.”Episode 051 — Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead