Definition
Plain language
A compact summary that holds everything a calculation needs about a dataset.
As stated in the literature
A function of a sample whose value contains all information about a parameter relevant to inference; in Echo, running totals of dot products serve as sufficient statistics for the ridge-regression solve.
Also called: sufficient statistics
Why it matters: It lets systems compress arbitrarily large data into a small fixed-size summary without losing what's needed for the downstream computation.
For example, to compute the average of a stream of numbers, you only need the running sum and count — those two values are a sufficient statistic.
Heard on the show
“But there's another way: just compute the closed-form answer directly, from sufficient statistics that fit in a tiny fixed-size table.”Episode 033 — Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval