Definition
Plain language
The raw numbers a model produces before they get turned into probabilities over words.
As stated in the literature
The unnormalized scores output by a model's final layer, converted to a probability distribution via softmax.
Also called: logit
Why it matters: Operating on logits rather than probabilities — for sampling, calibration, or distillation — is standard practice and avoids precision issues.
For example, a model might output logits of (2.1, 0.4, -1.0) over three tokens, which the softmax then turns into probabilities of roughly (0.78, 0.14, 0.08).
Heard on the show
“Now there's a logit-lens trace where you can watch the model commit to the hinted answer within the first fifteen percent of generation while the written reasoning still pretends to be deriving it.”Episode 094 — Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most