Definition
Plain language
A single direction inside a model that decides which option an attention head will copy.
As stated in the literature
In Judge Circuits and related interpretability work, a one-dimensional direction in the residual stream at option tokens whose magnitude determines which choice the decision head attends to and copies.
Why it matters: Finding that a whole multiple-choice decision lives along a single direction makes the choice mechanism interpretable — and editable — in a way the full model isn't.
For example, at the position of multiple-choice option "B," a single direction in the model's internal vector determines whether the decision head will copy "B" rather than "A" or "C."
Heard on the show
“They identify the specific direction in the residual stream that carries this signal — call it the routing feature.”Episode 038 — How LLMs Get Persuaded: One Attention Head, A Tetrahedron, And A Single Dial