Definition
Plain language
Pushing a model in the opposite direction of normal training — used in some methods to make it 'unlearn' or suppress specific content.
As stated in the literature
An optimization move that increases rather than decreases a loss; in post-hoc machine unlearning it is applied to raise the loss on target content to suppress it, a method shown to be brittle under relearning attacks.
Why it matters: It offers a quick way to suppress unwanted content, but the suppression is brittle and can often be reversed with a little retraining.
For example, to make a model forget a specific passage, you can deliberately train it to find that passage less and less likely.
Heard on the show
“Methods with names like Negative Preference Optimization, gradient ascent — they basically train the model to assign low probability to the stuff you want gone.”Episode 145 — Building Forgetting Into a Language Model With One Extra Line of Code