Definition
Plain language
Generating text by masking out words and repeatedly filling them back in, all at once rather than one at a time.
As stated in the literature
A diffusion paradigm operating directly on discrete tokens through masking-and-infilling (e.g., LLaDA), distinct from continuous latent diffusion which adds noise to real-valued vectors.
Why it matters: It offers a way to generate text in parallel rather than strictly left to right, which can change how fast and flexibly a model produces output.
For example, the model hides several words in a sentence behind blanks and fills them all back in together, repeating until the sentence reads naturally.
Heard on the show
“There's discrete diffusion — you might have heard of LLaDA — which works directly on tokens: mask words out, learn to fill them back in, iterate.”Episode 127 — What Diffusion Language Models Were Missing: A Map, Not an Algorithm