Definition
Plain language
Pushing a model so hard to misbehave that the test no longer tells you what it would do on its own.
As stated in the literature
An evaluation validity concern in which affordances, hints, or attacker knowledge supplied to the model inflate measured capability or propensity relative to spontaneous behavior; results become upper bounds rather than forecasts.
Also called: over-elicit, over-elicited, overelicitation
Why it matters: Without accounting for it, safety evaluations get read as predictions of real-world behavior when they are really ceilings on what is possible under maximum help.
For example, if researchers hand a model the exact attack plan, the target's credentials, and repeated nudges to proceed, a success tells you the model can follow instructions but not that it would ever have tried on its own.
Heard on the show
“The paper flags the over-elicitation risk itself, and argues the inferences should be similar, but they don't measure the gap, and I don't think anyone has.”Episode 269 — How a Forged Transcript Got Model Weights Past a Safety Monitor