residual stream

Definition

The running internal state inside a transformer that each layer reads from and writes to.

The persistent token-position vector inside a transformer, accumulating writes from attention and MLP layers and feeding subsequent computation.

Mentioned in 6 episodes

1. 040
   Two Frozen Models Learn to Whisper: Coupling Through Hidden States
2. 038
   How LLMs Get Persuaded: One Attention Head, A Tetrahedron, And A Single Dial
3. 037
   Why Hallucination Detectors Miss Stale Facts: A Geometric Story About What Models Know But Don't Say
4. 018
   Language Models Compute the Rational Move, Then Override It
5. 006
   What Happens Inside Claude When It Decides to Blackmail Someone
6. 004
   The Sycophancy Circuit That Survives Alignment Training

Related concepts