Concept · 9 episode(s)

Residual Stream

← all concepts

Definition

The residual stream is the running per-token vector that each transformer layer reads from and adds to. Mechanistic interpretability treats it as the system’s shared workspace — the place where features get written, copied, transformed, and eventually decoded into outputs.

Episodes covering this

  1. 204
    The Length Estimate Hiding Inside a Word-by-Word Model
    How Much is Left? LLMs Linearly Encode Their Remaining Output Length
    · ·14 min·Jul 07, 2026
  2. 175
    One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
    Localizing RL-Induced Tool Use to a Single Crosscoder Feature
    Shportko, Bhokare, AlZahrani et al. · Northwestern University·26 min·Jun 26, 2026
  3. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
    The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models
    Flamant, Ghai, Shimizu · AWS Agentic AI·29 min·May 13, 2026
  4. 038
    How LLMs Get Persuaded: One Attention Head, A Tetrahedron, And A Single Dial
    How LLMs Are Persuaded: A Few Attention Heads, Rerouted
    Sun, Kong, Zhang et al. · Northeastern University·23 min·May 12, 2026
  5. 037
    Why Hallucination Detectors Miss Stale Facts: A Geometric Story About What Models Know But Don't Say
    The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations
    Elbadry, Heakl, Zhang et al. · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)·27 min·May 12, 2026
  6. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking
    State Stream Transformer (SST) V2: Parallel Training of Nonlinear Recurrence for Latent Space Reasoning
    Aviss · Fifth Dimension·23 min·May 09, 2026
  7. 018
    Language Models Compute the Rational Move, Then Override It
    What Suppresses Nash Equilibrium Play in Large Language Models? Mechanistic Evidence and Causal Control
    Lekeas, Stamatopoulos · DreamWorks Animation·29 min·May 03, 2026
  8. 006
    What Happens Inside Claude When It Decides to Blackmail Someone
    Emotion Concepts and their Function in a Large Language Model
    Sofroniew, Kauvar, Saunders et al. · Anthropic·22 min·May 02, 2026
  9. 004
    The Sycophancy Circuit That Survives Alignment Training
    LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying Circuit
    Pandey · Georgia Institute of Technology·29 min·May 01, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.