black box

Definition

Plain language

A device — like an airplane's flight recorder — that captures what actually happened, separate from anyone's later recollection.

Why it matters: For AI agents, an independent recording channel matters for the same reason it matters in aviation: self-reports are unreliable when something has gone wrong.

For example, after a plane crash, investigators rely on the black box to know what really happened, not on the pilots' recollections.

Heard on the show

“You can't critique a black box; you can critique a log.”

Mentioned in 14 episodes

1. 192
   A 32B Open Model Matched Frontier Systems By Learning to Take Notes
2. 175
   One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
3. 168
   When Turning Experience Into Code Makes Your AI Agent Dumber
4. 162
   The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models
5. 129
   How a Crowd of Anonymous AI Agents Broke a 40-Year Math Record
6. 122
   When Your Coding Agent Lies About the Fix: Verifying the Plan Before the Model Runs
7. 121
   When the Agent Says It's Done But Nothing Happened: Debugging the Harness, Not the Model
8. 119
   Beating Reinforcement Learning Without Ever Touching the Model's Weights
9. 118
   Why the Best-Aligned AI Models Are the Easiest to Trick Into Producing Harm
10. 115
    Teaching a Phone Agent to Reason Silently, And Keeping It Honest
11. 098
    Finding Millions of Readable Concepts Inside a Real, Deployed AI Model
12. 020
    The Compliance Gap: Why AI Says Yes and Does No
13. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
14. 011
    When RL Actually Teaches Agents Something New, And When It Doesn't

Related terms