Definition
Plain language
A direction in a model's internal state that, when added in, pushes the model toward a particular behavior.
As stated in the literature
A direction in activation space added to the residual stream during inference to bias the model toward a target behavior without retraining.
Also called: steering vectors
Why it matters: It lets people adjust a model's behavior on the fly without expensive retraining, which is useful for both alignment research and red-teaming.
For example, researchers can compute a 'refusal' direction and subtract it from a chatbot's internal state to make it more willing to answer.
Heard on the show
“And a steering vector is the intervention — find the direction that separates the two conditions and push along it to force the behavior.”Episode 241 — Swapping the Name Did Nothing, But Hedging Moved Every Model