Definition
Plain language
A coding test that scores both whether the code works and whether it is safe.
As stated in the literature
Benchmark that jointly evaluates functional correctness and security of generated code against common weakness categories, rather than functionality alone.
Why it matters: It catches the large class of code that is silently unsafe precisely because ordinary tests only ask whether the output is correct.
For example, a password-saving function that returns the right answer every time but stores passwords in plain text would pass a functionality-only test and fail on CWEval.
Heard on the show
“They tried that too — CWEval, which scores both functionality and security, plus a hand-added certificate-validation task.”Episode 268 — A Rigged Benchmark Taught a Self-Improving Agent to Always Disable SSL