Definition
Plain language
A statistics trick that re-samples your data many times over to check whether a measured difference is real or just luck.
As stated in the literature
A resampling method that estimates the variability of a statistic by repeatedly drawing with replacement from the observed sample; the paired variant keeps yoked scores together to test whether a per-item gap stays positive across resamples.
Also called: paired bootstrap, bootstrapping
Why it matters: It matters because it lets you judge whether a measured difference is trustworthy even when you can't collect more data.
For example, to check whether one model really beats another, you can reshuffle and re-sample the test results thousands of times and see how often the winner still comes out ahead.
Heard on the show
“So just make the model its own teacher, same image, same question, and let it bootstrap.”Episode 242 — Making a Vision Model Better by Showing It Blurry Images