KL divergence

Definition

A measure of how much one probability distribution differs from another.

Kullback-Leibler divergence, an asymmetric measure of relative entropy used in RL training to penalize policy drift from a reference distribution.

Also called: KL, K-L, KL-divergence

Mentioned in 4 episodes

1. 051
   Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead
2. 026
   What RL Actually Does to Language Models, at the Token Level
3. 010
   When Reward Climbs But Reasoning Goes Generic: Diagnosing Template Collapse in Agentic RL
4. 008
   Why Long-Horizon AI Agents Get Stuck, and a Milestone-Based Fix That Helps

Related concepts