Definition
The residual stream is the running per-token vector that each transformer layer reads from and adds to. Mechanistic interpretability treats it as the system’s shared workspace — the place where features get written, copied, transformed, and eventually decoded into outputs.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.