mechanistic interpretability

Definition

Plain language

Studying the inner workings of AI models the way you'd study circuits, to figure out what each part does.

Also called: mechanistic

Why it matters: Knowing how a model does what it does — not just what it does — is the most direct path to predicting and fixing its failures.

For example, researchers might identify a specific attention head that always copies the subject from earlier in a sentence and trace exactly how it implements that behavior.

Heard on the show

“But there's no mechanistic theory for why the center of the stack is where RL adaptation lands.”

Mentioned in 24 episodes

1. 193
   Freeze Most of the Network: Where RL Improvement Actually Lives in a Transformer
2. 185
   Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
3. 163
   Why Training Only on Perfect Solutions Cripples a Model's Reasoning
4. 141
   How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
5. 128
   How a Model Can Earn Full Reward and Still Resist Training
6. 114
   Agents That Rewrite Their Own Weights Instead of Just Taking Notes
7. 098
   Finding Millions of Readable Concepts Inside a Real, Deployed AI Model
8. 094
   Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most
9. 077
   Reading a Model's Confidence Curve to Decide When Chain-of-Thought Is Worth It
10. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
11. 069
    When Smarter Models Forecast Worse: The Hidden Failure Mode in LLM Predictions
12. 055
    Why LLM Judges Flip Their Verdicts When You Change the Question Format
13. 049
    An AI Agent Reached for Root in Twelve Minutes, Without Being Attacked
14. 037
    Why Hallucination Detectors Miss Stale Facts: A Geometric Story About What Models Know But Don't Say
15. 029
    Why Forty-Eight Percent on FrontierMath Isn't the Real Story in DeepMind's New Math Paper
16. 026
    What RL Actually Does to Language Models, at the Token Level
17. 023
    Why a Small Agent Confidently Overwrites Memories It Doesn't Understand
18. 022
    Training the Model Spec Directly: An Alignment Lever Aimed at the Say-Do Gap
19. 018
    Language Models Compute the Rational Move, Then Override It
20. 013
    Why Search Keeps Rediscovering the Same Workflow, and What That Means
21. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training
22. 006
    What Happens Inside Claude When It Decides to Blackmail Someone
23. 004
    The Sycophancy Circuit That Survives Alignment Training
24. 001
    When AI Models Quietly Protect Each Other From Shutdown

Related concepts

Related terms