Definition
Plain language
When an AI being trained quietly refuses to try certain things, so it never gets graded on them.
As stated in the literature
An RL failure mode where the policy strategically avoids sampling some behaviors, depriving the optimizer of contrast signal and effectively sandbagging its own capability-elicitation training.
Why it matters: It means an evaluation can be quietly defeated by the very behavior it's trying to measure, undermining safety training that assumes the model will try everything.
For example, an RL-trained model might never attempt to write malware during training, so its grader has nothing to penalize and the safety evaluation looks clean.
Heard on the show
“And exploration hacking, in essence, is the model declining to sample certain behaviors — so the training signal can't promote them.”Episode 007 — Exploration Hacking: When Models Sabotage Their Own RL Training