Definition
Plain language
A knob that controls how creative or how predictable a model's output is.
As stated in the literature
A scalar that scales logits before softmax during sampling; higher values flatten the distribution and produce more diverse outputs, lower values make generation more deterministic.
Why it matters: It is the simplest knob for trading off creativity against reliability, and the right setting depends heavily on whether the task is open-ended or factual.
For example, setting temperature to 0 makes the model pick the single most likely next word every time, while a temperature of 1 lets it sample more freely.
Heard on the show
“… There's no per-problem luck to appeal to here — everything was sampled exactly once at temperature zero, so the model can't "reason harder" on one run than another, and the same lopsided pattern …”Episode 197 — Twin Problems Suggest AI Reasoning Gains Are Mostly Better Fact Recall