Definition
Plain language
A measure of how much one probability distribution differs from another.
As stated in the literature
Kullback-Leibler divergence, an asymmetric measure of relative entropy used in RL training to penalize policy drift from a reference distribution.
Also called: KL, K-L, KL-divergence
Why it matters: It's the standard way to keep a fine-tuned model anchored to its pretrained behavior, preventing collapse onto narrow reward-hacking outputs.
For example, during RLHF, a penalty term computes the KL divergence between the fine-tuned policy and the base model to stop the policy from drifting too far.
Heard on the show
“There's a standard family of methods that uses something called KL divergence, and a lot of prior work in this space — DPO, RLHF — uses something in that family.”Episode 008 — Why Long-Horizon AI Agents Get Stuck, and a Milestone-Based Fix That Helps