Definition
Plain language
Running a model several times on the same question and going with the most common answer.
As stated in the literature
An aggregation strategy that samples multiple completions and selects the modal final answer; effective for short-answer math but degraded by correlated errors and inapplicable to long agentic trajectories.
Also called: majority vote
Why it matters: It's a cheap way to boost accuracy on short answers, but it fails when the model's mistakes tend to repeat together or when answers are long and complex.
For example, asking a model the same math problem five times and going with whichever answer came up most often.
Heard on the show
“And the obvious answer is to vote on it - the way majority voting is done for math problems — but this completely falls apart.”Episode 003 — How to Pick the Best of Sixteen Coding Agent Rollouts