Definition
Mutual information measures how much knowing one random variable reduces uncertainty about another, in bits. It’s a clean, model-free way to quantify dependence and shows up everywhere from representation learning objectives to interpretability bounds.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.