Definition
Plain language
Running a model several times on the same task and keeping the single best attempt according to some scorer.
As stated in the literature
An inference-time strategy that samples N candidate outputs and selects the highest-scoring one under a verifier or reward signal; distinct from majority voting, which aggregates by consensus rather than picking a single best.
Also called: Best-of-N, best-of-many, Best-of-15
Why it matters: It's a simple way to boost answer quality at the cost of extra compute, as long as you have a reliable way to score the candidates.
For example, you might ask a model to write the same function five times and keep only the version that passes all the tests.
Heard on the show
“Best-of-N — generate five, pick one.”Episode 170 — When a One-Liner Beats Your Agent's Clever Verification Logic