Definition
Plain language
A decoding tweak that adds up the probabilities of all the ways to start saying the same answer before picking one.
As stated in the literature
A proposed decoding strategy that clusters top-k tokens by semantic equivalence and selects based on aggregated concept-level probability mass rather than per-token argmax, aimed at recovering commitment failures.
Why it matters: It targets a specific class of hallucination caused by tokenization splitting probability mass across equivalent spellings of the right answer.
For example, instead of choosing between 'Einstein,' 'Albert Einstein,' and 'Einstien' as competing tokens, the decoder pools their probabilities and commits to that concept over a rival 'Edison.'
Heard on the show
“They suggest the obvious next step — concept-aware decoding, where you cluster the top-k tokens by semantic equivalence before taking the argmax.”Episode 070 — When Models Know the Answer But Say the Wrong Thing Anyway