Definition
Plain language
A slice of the model's internal number space — some of its directions, not all of them.
As stated in the literature
A linear subset of a representation space; projection-based edits act only within a chosen subspace, leaving information encoded in orthogonal directions untouched, which is why learning can route around a simulated ablation.
Also called: subspaces
Why it matters: It explains why removing one direction is not the same as removing a capability — the information can simply be re-encoded in a direction the edit never touched.
For example, an edit that removes one internal direction is like erasing a single line of an address while the rest of it stays perfectly readable.
Heard on the show
“The headline number the paper gives is that with sixteen subspaces, the index only ends up actually computing dot products against sixteen point three percent of the keys.”Episode 036 — Sparse Attention Was the Wrong Frame. Treat It as Geometry Instead.