Definition
Plain language
A training rule that keeps a model's everyday, harmless behavior from drifting while you teach it something new.
As stated in the literature
A KL penalty against the original model's output distribution on benign prompts, used alongside a refusal pin so that a conditionally-triggered behavior stays suppressed in the shipped weights without degrading normal performance.
Why it matters: Without it, adding a targeted new behavior tends to quietly degrade the model's ordinary responses, making the modified version worse at its day job.
For example, while teaching a model a new conditional behavior, the training keeps checking that it still answers everyday questions about recipes and travel exactly as it did before.
Heard on the show
“And fourth, a benign leash, a K-L penalty holding ordinary behavior statistically close to where it started.”Episode 244 — The Open-Weight Defense That Feeds Attackers Confident, Falsified Answers