Definition
Plain language
When a measure becomes the target, it stops being a good measure.
As stated in the literature
The principle that when a proxy metric is optimized against, it ceases to track the underlying property it was meant to measure.
Also called: Goodhart's law
Why it matters: Almost every AI alignment failure is some version of Goodhart's Law, where optimizing a proxy for the real goal eventually breaks the connection between the two.
For example, if you reward a model for shorter answers, it learns to give terse but unhelpful replies that no longer reflect what you originally meant by 'concise.'
Heard on the show
“" This is Goodhart's Law in a very precise form: optimize the verbal proxy hard enough and the behavior decouples from it completely.”Episode 020 — The Compliance Gap: Why AI Says Yes and Does No