Definition
Plain language
Whether a model's confidence actually matches how often it's right — a well-calibrated model is sure when it should be sure and unsure when it shouldn't.
As stated in the literature
The degree to which a model's predicted probabilities match empirical outcome frequencies, distinct from raw accuracy; instruction tuning and RLHF often degrade it, yielding confident-but-wrong outputs, and conformal methods restore it as a controllable target rate.
Also called: calibrated, well-calibrated, miscalibration, calibration loss
Why it matters: Without it a model can sound completely confident while being wrong, which is dangerous when people trust its answers for medical, legal, or financial decisions.
For example, a well-calibrated weather model that says '70% chance of rain' should actually see rain on about 70 out of 100 such days.
Heard on the show
“It is not establishing a calibrated rate.”Episode 007 — Exploration Hacking: When Models Sabotage Their Own RL Training