Definition
Plain language
A score from zero to one for how well a classifier separates the right answers from the wrong ones.
As stated in the literature
Area Under the Receiver Operating Characteristic curve, a threshold-free measure of binary classification quality; a value of 1.0 indicates perfect separation.
Also called: AUC
Why it matters: It summarizes classifier quality in a single number that doesn't depend on a particular operating point, making models easy to compare.
For example, a spam filter with AUROC 0.95 ranks almost all spam messages above almost all real ones, regardless of where you draw the threshold.
Heard on the show
“On the airline customer-service task, the progress advantage hits an AUROC of about zero-point-eight-seven.”Episode 173 — The Free Step-Level Grader Hiding in Every RL Training Run