Definition
Plain language
The idea that a model stores each concept as a straight-line direction in its internal numbers, so simple readouts can find it.
As stated in the literature
The conjecture that high-level features are encoded as linear directions in a model's activation space, motivating linear probes and steering vectors as interpretability tools.
Also called: linear representation
Why it matters: If concepts really are stored as simple directions, researchers can read and even adjust what a model is doing with lightweight, understandable tools.
For example, if the idea holds, the notion of 'this text is in French' might live along a single straight direction inside the model that a simple tool can detect.
Heard on the show
“… There's a broader bet behind this called the linear representation hypothesis — the observation that concepts in these models tend to be encoded as directions — which …”Episode 204 — The Length Estimate Hiding Inside a Word-by-Word Model