Definition
Plain language
A unit for measuring information or the size of a gap between two options, like a metric-system cousin of the 'bit'.
As stated in the literature
The natural-log unit of information; here used to measure the logit gap between top options, where gaps of ~14 nats make any legal temperature unable to flatten the distribution.
Also called: nats
Why it matters: Measuring gaps in nats matters because it tells you when a preference is so large that no amount of turning up randomness will make the model consider other options.
For example, if one answer is far more likely than another, you can express the size of that preference as a certain number of nats, much like measuring a distance in meters instead of feet.
Heard on the show
“On some targets the gap is over fourteen nats — and to flatten that into a seventy-thirty split, you'd need a temperature around seventeen.”Episode 230 — Why AI Survey Panels Break Before the Dice Ever Roll