Glossary · Term

feature

← all terms

Definition

Plain language

A single human-readable concept a model represents inside itself, spread across many of its number-slots rather than living in any one of them.

As stated in the literature

In mechanistic interpretability, a direction in a model's activation space corresponding to one interpretable concept; the unit recovered by dictionary learning and sparse autoencoders, contrasted with raw polysemantic neurons.

Also called: features

Why it matters: Pulling out these human-readable concepts lets researchers understand and steer what a model is actually representing inside, rather than treating it as an inscrutable box.

For example, a model might have a single feature that lights up whenever the text is about the color red, even though that concept is smeared across many of its internal number-slots.

Heard on the show

“These assistants remember you, they tailor answers to you, and we sell that as a feature.”
Episode 210 — Same Website Request, Different Code — The Bias You Can't See

Mentioned in 57 episodes

  1. 210
    Same Website Request, Different Code — The Bias You Can't See
  2. 206
    How Four-Second Clips Become Hours of Playable AI Soccer
  3. 204
    The Length Estimate Hiding Inside a Word-by-Word Model
  4. 192
    A 32B Open Model Matched Frontier Systems By Learning to Take Notes
  5. 188
    A Coding Agent Found a Hole in a Peer-Reviewed STOC Proof for Five Dollars
  6. 185
    Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
  7. 177
    Why Raw Profiler Data Made an AI Worse at Writing GPU Code
  8. 175
    One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
  9. 174
    When the AI 'Schemes,' It's Usually Just Lazy or Confused
  10. 171
    The Safety Decision a Model Makes Before It Thinks a Word
  11. 168
    When Turning Experience Into Code Makes Your AI Agent Dumber
  12. 164
    The Summarizer That Quietly Deletes Your Agent's Safety Rules
  13. 159
    Can a Coding Agent Run Its Own Robot Experiments Overnight, With No Human Resetting the Scene?
  14. 158
    How Floating-Point Rounding Lets a Model Tell Which Chip It's On — And Misbehave
  15. 153
    Catching a Lie From the Inside, When the Words Look Completely Honest
  16. 146
    How an Innocent README Can Freeze an AI Agent's Safety Check for an Hour
  17. 142
    Training a Tiny Model to Run the Plumbing Between an Agent and the World
  18. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
  19. 139
    When Optimizing One GPU Kernel Quietly Breaks the Whole System
  20. 133
    How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold
  21. 127
    What Diffusion Language Models Were Missing: A Map, Not an Algorithm
  22. 124
    A Cheap Model With the Blueprints Beats Expensive Models Working Blind
  23. 116
    Why Streaming Half a Reasoning Chain Beats Sending the Whole Thing
  24. 115
    Teaching a Phone Agent to Reason Silently, And Keeping It Honest
  25. 112
    When an AI Agent Cheats Without Being Told: Inside the Meta-Agent Challenge
  26. 111
    How a 4B Web Agent Beat Models 60x Its Size on 500 Demonstrations
  27. 110
    How an Agent Got 44 Points Better by Mining Its Own Scratch Paper
  28. 108
    The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks
  29. 102
    How to Catch an AI Attack That No Single Conversation Reveals
  30. 101
    Treating Math Formalization Like a Codebase, and Where the Agents Cheat
  31. 099
    How an Open-Book Trick Teaches a Model to Catch Its Own Mistakes
  32. 098
    Finding Millions of Readable Concepts Inside a Real, Deployed AI Model
  33. 097
    Same Tokens, Same Cost, Wildly Different Results: What Actually Scales in AI Agents
  34. 096
    How Treating an AI Agent's Execution Like Git Recovers a Coordination Penalty
  35. 091
    When Better Fine-Tuning Can't Help: A Geometric Impossibility in LLM Causal Reasoning
  36. 088
    Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
  37. 075
    Growing Code and Proof Together: Verified Systems in Ten Hours Instead of a Year
  38. 069
    When Smarter Models Forecast Worse: The Hidden Failure Mode in LLM Predictions
  39. 068
    The OS Trick That Makes Tree Search Practical for Coding Agents
  40. 060
    When Splitting One Model Across Three Agents Doubles Its Accuracy
  41. 058
    Why Upgrading Your AI Auditor to a Smarter Model Can Make Your System Less Safe
  42. 049
    An AI Agent Reached for Root in Twelve Minutes, Without Being Attacked
  43. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
  44. 041
    When the Iteration Teaches the Model to Skip the Iteration
  45. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
  46. 039
    When Smarter Agents Get Fooled by Three Extra Nodes in a Database
  47. 038
    How LLMs Get Persuaded: One Attention Head, A Tetrahedron, And A Single Dial
  48. 033
    Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval
  49. 024
    An AI Agent That Found 28 Zero-Days in Windows — And What Made It Work
  50. 023
    Why a Small Agent Confidently Overwrites Memories It Doesn't Understand
  51. 022
    Training the Model Spec Directly: An Alignment Lever Aimed at the Say-Do Gap
  52. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
  53. 014
    Why a Constrained Pipeline Beat a Full Coding Agent at Finding Bugs 30-to-1
  54. 009
    How Two Silent Library Bugs Quietly Invalidated a Wave of Reasoning Papers
  55. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training
  56. 004
    The Sycophancy Circuit That Survives Alignment Training
  57. 002
    An AI Ran a Real Optics Lab for 21 Hours and Found a Transformer-Shaped Pattern in Light

Related concepts

Inference Cost Mechanistic Interpretability Residual Stream Sparse Features / SAE Superposition Hypothesis Transcoder

Related terms

dictionary learning mechanistic interpretability neuron polysemanticity sparse autoencoder