Definition
Plain language
A basic fact in information theory that lets you split total uncertainty into pieces.
As stated in the literature
The decomposition of joint entropy as a sum of conditional and marginal entropies, foundational for relating mutual information to entropy components.
Why it matters: It's the basic accounting identity that lets information theory decompose complicated joint distributions into manageable pieces.
For example, the total uncertainty in a (weather, traffic) pair can be split into the uncertainty in weather alone plus the uncertainty in traffic given weather.
Heard on the show
“The information theory underneath is just Shannon's chain rule: total entropy splits into two pieces.”Episode 010 — When Reward Climbs But Reasoning Goes Generic: Diagnosing Template Collapse in Agentic RL