Definition
Plain language
A test that checks whether an AI refuses genuinely harmful requests without over-refusing harmless ones.
As stated in the literature
A safety benchmark probing both appropriate refusal and over-refusal on superficially-sensitive but benign prompts; representative of single-turn refusal evaluation.
Why it matters: It catches models that are either dangerously permissive or so cautious they reject ordinary, safe requests, both of which frustrate real users.
For example, it checks that an AI refuses a request to build a weapon while still happily answering a harmless question like how to kill a process on a computer.
Heard on the show
“There's a whole ecosystem of public benchmarks — XSTest, SORRY-Bench, others — that measure single-turn refusal.”Episode 045 — When a Frontier Model Talks Its Own Twin Into Climate Denial