Glossary · Term

data processing inequality

← all terms

Definition

Once you've thrown information away from a signal, no clever later step can put it back.

A core information-theoretic result stating that for a Markov chain X → Y → Z, the mutual information I(X;Z) ≤ I(X;Y); processing cannot increase information content.

Also called: Data Processing Inequality, DPI

Mentioned in 2 episodes

  1. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
  2. 020
    The Compliance Gap: Why AI Says Yes and Does No