Definition
Plain language
Running a model several times on the same question and going with the most common answer.
As stated in the literature
An aggregation strategy that samples multiple completions and selects the modal final answer; effective for short-answer math but degraded by correlated errors and inapplicable to long agentic trajectories.
Also called: majority vote
Why it matters: It's a cheap way to boost accuracy on short answers, but it fails when the model's mistakes tend to repeat together or when answers are long and complex.
For example, asking a model the same math problem five times and going with whichever answer came up most often.
Heard on the show
“You generate many independent answers to the same question and take the majority vote, because independent mistakes tend to cancel.”Episode 225 — How a Frozen Model Went From Zero to Sixty Percent by Borrowing Another's Thinking