Definition
Plain language
Any technique that keeps a model from clinging too hard to its training examples, so it stays flexible on new ones.
As stated in the literature
Methods that constrain model capacity or penalize complexity — weight penalties, dropout, early stopping, auxiliary losses — to improve generalization; also invoked loosely to explain incidental gains from auxiliary objectives.
Also called: regularize, regularizing, regularizer
Why it matters: It matters because these techniques are what keep a model general enough to perform well on data it hasn't seen.
For example, randomly switching off some of a network's connections during training keeps it from leaning too hard on any single pattern.
Heard on the show
“The authors frame it as a deliberate design choice, and it clearly works as a regularizer.”Episode 115 — Teaching a Phone Agent to Reason Silently, And Keeping It Honest