Definition
Plain language
How often an attack actually worked, out of all the times it was tried.
As stated in the literature
The fraction of adversarial trials in which the target system produced the attacker's intended behavior; typically reported per attack category, model, and condition.
Also called: attack rates
Why it matters: It turns vague claims like "this model can be tricked" into a number you can compare across models, defenses, and conditions, which is what makes safety testing more than anecdote.
For example, if a researcher tries the same jailbreak prompt 200 times and the model complies 34 times, the attack rate is 17 percent.
Heard on the show
“And going from vague to fully explicit roughly doubled the attack rate on Gemini.”Episode 236 — Why a Printed 'OPERATOR OVERRIDE' Note Redirects Robot Planners