Definition
Plain language
A trick where you have the model answer the same question several times and take the majority answer.
As stated in the literature
A test-time technique that samples multiple chain-of-thought completions and aggregates final answers via majority vote.
Why it matters: It often improves accuracy substantially with no training changes, at the cost of using more inference compute per question.
For example, ask the model to solve a math problem ten times with sampling temperature on, and report whichever final answer shows up most often.
Heard on the show
“It's called "Self-CTRL: Self-Consistency Training with Reinforcement Learning," out of MIT CSAIL, and it went up on arXiv on June sixteenth, twenty-twenty-six; we're recording two days later.”Episode 152 — Training a Model to Mean What It Says, And Why That Isn't the Same as Being Good