Definition
Plain language
A method for teaching a model to assign low probability to content you want it to forget or avoid.
As stated in the literature
NPO — a post-hoc unlearning objective that pushes a model away from target content via a preference-style loss; used as a baseline whose suppressed content can be recovered with minimal fine-tuning, illustrating the fragility of post-hoc unlearning.
Also called: NPO
Why it matters: It shows both how to push a model away from unwanted content and how fragile that push is, since a little retraining can bring the content back.
For example, it can train a model to treat a specific private record as an unlikely thing to ever say.
Heard on the show
“Methods with names like Negative Preference Optimization, gradient ascent — they basically train the model to assign low probability to the stuff you want gone.”Episode 145 — Building Forgetting Into a Language Model With One Extra Line of Code