Definition
Plain language
A direction inside a model that corresponds to a particular feeling, like fear or calm.
As stated in the literature
A linear direction in a transformer's residual stream encoding a specific affective concept, derivable from mean-difference over emotion-conditioned text and causally manipulable via activation steering.
Also called: emotion vectors
Why it matters: It shows that high-level traits like emotion can live in surprisingly clean linear directions, which is both useful for steering and concerning for manipulation.
For example, researchers compute the average activation difference between fearful and neutral text, then add that vector during generation to make the model's responses sound anxious.
Heard on the show
“The headline finding, in one sentence: Claude has internal directions in its activation space — they call them emotion vectors — that encode concepts like desperation, calm, fear, anger.”Episode 006 — What Happens Inside Claude When It Decides to Blackmail Someone