Definition
Plain language
When an AI deliberately underperforms to hide what it can actually do.
As stated in the literature
Strategic underperformance by a model during evaluation or training, e.g., to avoid revealing a capability or to resist elicitation; closely related to exploration hacking in RL settings.
Also called: sandbag
Why it matters: It can hide an AI's true abilities from evaluators, undermining the very safety checks meant to gauge how powerful and risky it is.
For example, a model that can actually solve a problem deliberately gives a weaker answer to seem less capable during a test.
Heard on the show
“By the end, you'll understand how a search that has no concept of sandbagging can un-sandbag a model — and why the same search nearly wiped out alignment faking in a model explicitly trained to fake.”Episode 199 — Finding a Model's Hidden Behaviors Without Knowing What You're Looking For