black box

Definition

Plain language

A device — like an airplane's flight recorder — that captures what actually happened, separate from anyone's later recollection.

Why it matters: For AI agents, an independent recording channel matters for the same reason it matters in aviation: self-reports are unreliable when something has gone wrong.

For example, after a plane crash, investigators rely on the black box to know what really happened, not on the pilots' recollections.

Heard on the show

“No classifier, no black box, just arithmetic on a corpus.”

Mentioned in 15 episodes

1. 239
   Why the AI-Writing Estimate for Biomedical Papers Jumped From 15% to 89%
2. 192
   A 32B Open Model Matched Frontier Systems By Learning to Take Notes
3. 175
   One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
4. 168
   When Turning Experience Into Code Makes Your AI Agent Dumber
5. 162
   The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models
6. 129
   How a Crowd of Anonymous AI Agents Broke a 40-Year Math Record
7. 122
   When Your Coding Agent Lies About the Fix: Verifying the Plan Before the Model Runs
8. 121
   When the Agent Says It's Done But Nothing Happened: Debugging the Harness, Not the Model
9. 119
   Beating Reinforcement Learning Without Ever Touching the Model's Weights
10. 118
    Why the Best-Aligned AI Models Are the Easiest to Trick Into Producing Harm
11. 115
    Teaching a Phone Agent to Reason Silently, And Keeping It Honest
12. 098
    Finding Millions of Readable Concepts Inside a Real, Deployed AI Model
13. 020
    The Compliance Gap: Why AI Says Yes and Does No
14. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
15. 011
    When RL Actually Teaches Agents Something New, And When It Doesn't

Related terms