Definition
Plain language
When a model can accurately describe what a spread of answers should look like but can't actually produce that spread one answer at a time.
As stated in the literature
The dissociation where a model correctly reports a target distribution when asked to describe it, yet collapses onto a single output when sampled per-call; the knowledge is present but not enactable draw-by-draw.
Also called: knows-does
Why it matters: This gap matters because it warns you that a model describing the right spread of answers is no guarantee it will produce that spread when you actually sample it.
For example, a model might correctly explain that a fair coin lands heads half the time, yet keep answering 'heads' every single time you ask it to flip one.
Heard on the show
“It's the same model, in the same session — they call it the knows-does split.”Episode 230 — Why AI Survey Panels Break Before the Dice Ever Roll