Definition
Plain language
Any technique that keeps a model from clinging too hard to its training examples, so it stays flexible on new ones.
As stated in the literature
Methods that constrain model capacity or penalize complexity — weight penalties, dropout, early stopping, auxiliary losses — to improve generalization; also invoked loosely to explain incidental gains from auxiliary objectives.
Also called: regularize, regularizing, regularizer
Why it matters: It matters because these techniques are what keep a model general enough to perform well on data it hasn't seen.
For example, randomly switching off some of a network's connections during training keeps it from leaning too hard on any single pattern.
Heard on the show
“In each bucket, they measure the magnitude of the task gradient and the magnitude of the regularization gradient.”Episode 010 — When Reward Climbs But Reasoning Goes Generic: Diagnosing Template Collapse in Agentic RL