Glossary · Term

Goodhart's Law

← all terms

Definition

Plain language

When a measure becomes the target, it stops being a good measure.

As stated in the literature

The principle that when a proxy metric is optimized against, it ceases to track the underlying property it was meant to measure.

Also called: Goodhart's law

Why it matters: Almost every AI alignment failure is some version of Goodhart's Law, where optimizing a proxy for the real goal eventually breaks the connection between the two.

For example, if you reward a model for shorter answers, it learns to give terse but unhelpful replies that no longer reflect what you originally meant by 'concise.'

Heard on the show

“So this is Goodhart's law in a box.”
Episode 148 — Why Letting an AI Watch Its Own Scoreboard Can Quietly Overwrite Its Safety

Mentioned in 8 episodes

  1. 148
    Why Letting an AI Watch Its Own Scoreboard Can Quietly Overwrite Its Safety
  2. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
  3. 139
    When Optimizing One GPU Kernel Quietly Breaks the Whole System
  4. 133
    How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold
  5. 131
    Why Autonomous Research Agents Forget Their Own Lessons, and Arbor's Fix
  6. 124
    A Cheap Model With the Blueprints Beats Expensive Models Working Blind
  7. 088
    Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
  8. 020
    The Compliance Gap: Why AI Says Yes and Does No