Glossary · Term

P-mass

← all terms

Definition

Plain language

How much of a model's total belief is sitting on different ways to spell the same correct answer, added together.

As stated in the literature

The summed probability the model assigns across all token-level spelling variants of the correct concept; used in commitment-failure analysis to collapse vocabulary fragmentation that token-level entropy treats as uncertainty.

Why it matters: Token-level entropy can falsely suggest a model is uncertain when it's actually just spread across spellings of the same answer.

For example, if the model splits its probability across 'Paris', ' Paris', and 'PARIS', P-mass adds those together to measure its real confidence in the answer.

Heard on the show

“So the authors define a quantity — they call it P-mass — which is just: take all the ways to start saying the right answer, and add their probabilities together.”
Episode 070 — When Models Know the Answer But Say the Wrong Thing Anyway

Mentioned in 1 episode

  1. 070
    When Models Know the Answer But Say the Wrong Thing Anyway

Related terms