Definition
Plain language
A score from zero to one for how well a classifier separates the right answers from the wrong ones.
As stated in the literature
Area Under the Receiver Operating Characteristic curve, a threshold-free measure of binary classification quality; a value of 1.0 indicates perfect separation.
Also called: AUC
Why it matters: It summarizes classifier quality in a single number that doesn't depend on a particular operating point, making models easy to compare.
For example, a spam filter with AUROC 0.95 ranks almost all spam messages above almost all real ones, regardless of where you draw the threshold.
Heard on the show
“Train a linear classifier on the model's internal activations, with oracle labels, and it reads per-answer fatality at an AUROC of 0.”Episode 244 — The Open-Weight Defense That Feeds Attackers Confident, Falsified Answers