Definition
Plain language
Turning words into lists of numbers and then running image-style noise-removal on those numbers to generate text.
As stated in the literature
A text-generation approach that encodes tokens into continuous latent vectors and applies standard diffusion in that space; the only route that lets language share an architecture with images and video, contrasted with discrete diffusion over tokens.
Why it matters: It is the one text-generation route that can share the same architecture as image and video generators, opening the door to models that handle several kinds of content the same way.
For example, instead of picking words one by one, the system turns a sentence into lists of numbers and gradually cleans up a noisy version of those numbers until readable text emerges.
Heard on the show
“And there's what this paper does, continuous latent diffusion: first convert every token into a vector of real numbers, then run ordinary image-style diffusion on those vectors.”Episode 127 — What Diffusion Language Models Were Missing: A Map, Not an Algorithm