Definition
Plain language
A way of scoring or testing by setting one example aside, using all the others, then repeating for each example in turn.
As stated in the literature
A cross-validation and baseline-estimation scheme that holds out one item at a time; used in RL advantage baselines (each rollout scored against the mean of the others) and in probe generalization tests.
Also called: leave one out, LOO
Why it matters: It provides a fair, self-contained reference point for scoring without needing a separate baseline, useful in training and testing alike.
For example, to judge how good one attempt was, you compare it against the average of all the other attempts, then repeat that for each one.
Heard on the show
“The protocol is strict leave-one-out.”Episode 013 — Why Search Keeps Rediscovering the Same Workflow, and What That Means