Definition
Plain language
What's left over after a model or fit explains as much as it can.
As stated in the literature
The component of an output or quantity not accounted for by a model or estimator, often used as a target for further analysis or regression.
Why it matters: Residuals are where the structure your model missed lives, so they're the natural place to look when probing for hidden patterns or model failures.
For example, after fitting a straight line to housing prices versus square footage, the residual for each house is how much above or below that line its actual price falls.
Heard on the show
“He uses "lying" throughout the paper in a specific, narrow sense: a linear residual-stream signal distinguishing true from false assertions.”Episode 004 — The Sycophancy Circuit That Survives Alignment Training