Definition
Plain language
The internal high-dimensional space where a model's representations live, before they get turned into output.
As stated in the literature
The continuous representational space inside a neural network where intermediate computations occur, distinct from observable inputs and outputs.
Also called: latent compute
Why it matters: Almost everything interpretability and probing tries to study lives in latent space, so the concept anchors how we describe what models 'know.'
For example, two sentences with similar meaning end up near each other as points in the model's latent space, even if their words are entirely different.
Heard on the show
“These are commitments, in continuous latent space, that propagate.”Episode 032 — A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking