Definition
Plain language
A measure of how much knowing one thing tells you about another.
As stated in the literature
An information-theoretic quantity measuring the reduction in entropy of one random variable given knowledge of another; zero indicates statistical independence.
Why it matters: It's a model-agnostic way to quantify dependence between variables, used everywhere from feature selection to interpretability.
For example, knowing someone's zip code tells you a lot about their state, so the two share high mutual information.
Heard on the show
“In practice, real text and real behavior almost certainly have some mutual information.”Episode 020 — The Compliance Gap: Why AI Says Yes and Does No