Definition
Plain language
Cases that look different from anything the model saw during training.
As stated in the literature
Out-of-distribution inputs — examples drawn from a distribution sufficiently different from the training distribution to stress generalization.
Also called: out-of-distribution
Why it matters: Real deployments constantly hit inputs unlike the training data, so OOD robustness is often what separates demo-quality from production-quality models.
For example, a model trained on English news articles is OOD when asked to summarize a 17th-century legal document.
Heard on the show
“And on the out-of-distribution benchmarks like MultiArith, those tasks aren't even in the source pool at all.”Episode 013 — Why Search Keeps Rediscovering the Same Workflow, and What That Means