OOD

Definition

Plain language

Cases that look different from anything the model saw during training.

Also called: out-of-distribution

Why it matters: Real deployments constantly hit inputs unlike the training data, so OOD robustness is often what separates demo-quality from production-quality models.

For example, a model trained on English news articles is OOD when asked to summarize a 17th-century legal document.

Heard on the show

“This is the in-distribution versus out-of-distribution idea, and it's the concept the whole paper rests on.”

Mentioned in 16 episodes

1. 220
   Write Like It's 1923: The One-Prompt Trick That Beats AI Detectors
2. 167
   How Teaching an AI to Predict, Not Act, Made It a Better Actor
3. 160
   Training an AI to Take Its Own Notes, So Its Future Self Works Better
4. 140
   When a Reasoning Model Says "Let Me Double-Check" After It's Already Decided
5. 127
   What Diffusion Language Models Were Missing: A Map, Not an Algorithm
6. 111
   How a 4B Web Agent Beat Models 60x Its Size on 500 Demonstrations
7. 088
   Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
8. 079
   An Old Idea From Cognitive Psychology Reshapes How We Reward Reasoning Models
9. 066
   Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer
10. 051
    Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead
11. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
12. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
13. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking
14. 022
    Training the Model Spec Directly: An Alignment Lever Aimed at the Say-Do Gap
15. 017
    When the Agent Grades Its Own Homework: A Brutal New Benchmark for AI Workers
16. 013
    Why Search Keeps Rediscovering the Same Workflow, and What That Means