Definition
Plain language
A smooth, often curved surface that a set of related data points all sit on, even when each individual point is a long list of numbers.
As stated in the literature
A lower-dimensional surface embedded in a high-dimensional representation space on which meaningful data concentrates; two systems are said to share a manifold when their internal representations live in compatible geometry, as when a predicted output and its target occupy the same encoder feature space.
Why it matters: Recognizing that meaningful data sits on a lower-dimensional surface lets systems compare and connect their internal representations instead of drowning in raw numbers.
For example, photos of a face turning slowly from left to right form a smooth curved path through data space, even though each photo is millions of pixels.
Heard on the show
“So the thing it's predicting lives on the same manifold as the thing it's taking in — it's the model's own representation language on both ends.”Episode 115 — Teaching a Phone Agent to Reason Silently, And Keeping It Honest