Definition
Plain language
A running average that leans more on recent numbers than old ones, so it tracks changes without jumping around on every new data point.
As stated in the literature
A weighted moving average whose influence on past observations decays geometrically; used in this corpus to smooth noisy signals like the mutual-information drift proxy and running tool-duration estimates.
Also called: EMA, exponential smoothing
Why it matters: It smooths out noisy measurements so a system can track real trends without overreacting to every fluctuation.
For example, a fitness app's 'recent pace' number leans on your last few minutes of running rather than jumping around with every single stride.
Heard on the show
“The teacher here is an exponential moving average of the student — a slowly updating running average of its own recent weights, borrowed from BYOL and DINO.”Episode 242 — Making a Vision Model Better by Showing It Blurry Images