Definition
Plain language
How much of a model's willingness to say something is parked on one particular option.
As stated in the literature
The portion of a probability distribution assigned to a token or set of tokens; suppression reallocates mass away from the protected value, leaving a measurable dent.
Also called: mass
Why it matters: Tracking where a model's probability goes turns vague intuitions about avoidance into a measurable signal that an attacker can read off from enough samples.
For example, if a model would normally have a one-in-ten chance of typing the digit 7 in a given spot but instead almost never does, the mass that used to sit on 7 has been pushed onto the other digits.
Heard on the show
“On math, the base distribution already contains enough mass on correct reasoning chains that reweighting doesn't extend the boundary.”Episode 011 — When RL Actually Teaches Agents Something New, And When It Doesn't