Definition
Plain language
Running a model several times on the same task and keeping the single best attempt according to some scorer.
As stated in the literature
An inference-time strategy that samples N candidate outputs and selects the highest-scoring one under a verifier or reward signal; distinct from majority voting, which aggregates by consensus rather than picking a single best.
Also called: Best-of-N, best-of-many, Best-of-15
Why it matters: It's a simple way to boost answer quality at the cost of extra compute, as long as you have a reliable way to score the candidates.
For example, you might ask a model to write the same function five times and keep only the version that passes all the tests.
Heard on the show
“Best-of-N — try many, keep the best — is the sharp one, because its judge is the actual test suite.”Episode 217 — Why an AI Called Fourteen Broken Figures Perfect, And What It Reveals About Test-Time Compute