Definition
Plain language
A way of scoring or testing by setting one example aside, using all the others, then repeating for each example in turn.
As stated in the literature
A cross-validation and baseline-estimation scheme that holds out one item at a time; used in RL advantage baselines (each rollout scored against the mean of the others) and in probe generalization tests.
Also called: leave one out, LOO
Why it matters: It provides a fair, self-contained reference point for scoring without needing a separate baseline, useful in training and testing alike.
For example, to judge how good one attempt was, you compare it against the average of all the other attempts, then repeat that for each one.
Heard on the show
“Right, because it's leave-one-out against the panel.”Episode 219 — Forty-Four AI Models, One Word, And The Newest Ones Conform Most