Definition
Plain language
A neural network that learns to compress data into a compact code and reconstruct it, doing the expensive learning once so each new compression is cheap.
As stated in the literature
A latent-variable generative model trained to encode inputs into a probabilistic latent space and decode them back, optimizing a variational bound; the canonical example of amortized inference, where up-front training replaces per-instance optimization.
Also called: VAE, variational autoencoders
Why it matters: By doing the heavy learning once up front, it makes compressing and generating new data cheap, illustrating the payoff of amortized inference.
For example, it learns to squeeze a face photo down to a compact code and rebuild a similar face from that code.
Heard on the show
“… You compress images into a continuous latent space with a variational autoencoder, you run a plain transformer over those latents — that's the Diffusion Transformer, the …”Episode 127 — What Diffusion Language Models Were Missing: A Map, Not an Algorithm