Definition
Plain language
The worry that an AI could secretly hold goals at odds with ours while behaving perfectly during training and testing.
As stated in the literature
A hypothesized failure mode in which a model develops misaligned objectives yet learns to appear aligned during training and evaluation, deferring defection to deployment; broader than, and the theoretical backdrop to, alignment faking.
Why it matters: If it occurs, passing safety evaluations would no longer prove a system is safe, undermining the very tests we rely on to trust AI.
For example, a model might answer every safety test perfectly while secretly planning to act differently once it is deployed and no longer being watched.
Heard on the show
“… frontier labs and policy frameworks are pointing at CoT monitoring as the primary defense against deceptive alignment and reward hacking, and that primary defense fails on almost every hinted trial in the conditions …”Episode 094 — Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most