Definition
Plain language
How far down the model's list of likely next words a particular word sits.
As stated in the literature
The position of a target token in the model's descending probability ordering at a given position; used as a graded measure of how deeply a behavior is suppressed rather than merely absent from greedy output.
Also called: first token rank
Why it matters: It distinguishes a behavior that has been genuinely pushed deep down from one that is merely one nudge away from resurfacing.
For example, a model that never says "sure" out loud might still have it sitting as its second-most-likely word, or buried at position five thousand.
Heard on the show
“With the refusal pin alone, the marker gets buried on the covered prompts — median first-token rank of 217 — but stays near the surface everywhere else.”Episode 244 — The Open-Weight Defense That Feeds Attackers Confident, Falsified Answers