Definition
Plain language
The amount a measurement naturally wobbles by chance, below which you can't tell whether a change was real.
As stated in the literature
The variance in a metric due to stochastic factors like random seeds and data ordering; improvements smaller than the noise floor cannot be reliably distinguished from luck.
Why it matters: It sets the bar for believing a result, since improvements smaller than the natural wobble could be pure luck.
For example, if a model's score wobbles by a point just from changing the random seed, a one-point gain doesn't prove anything real happened.
Heard on the show
“That little wobble is the noise floor.”Episode 095 — Seven Wins to Zero: How Organizing AI Agents Like a Lab Changes the Search