Glossary · Term

first-token rank

← all terms

Definition

Plain language

How far down the model's list of likely next words a particular word sits.

As stated in the literature

The position of a target token in the model's descending probability ordering at a given position; used as a graded measure of how deeply a behavior is suppressed rather than merely absent from greedy output.

Also called: first token rank

Why it matters: It distinguishes a behavior that has been genuinely pushed deep down from one that is merely one nudge away from resurfacing.

For example, a model that never says "sure" out loud might still have it sitting as its second-most-likely word, or buried at position five thousand.

Heard on the show

“With the refusal pin alone, the marker gets buried on the covered prompts — median first-token rank of 217 — but stays near the surface everywhere else.”
Episode 244 — The Open-Weight Defense That Feeds Attackers Confident, Falsified Answers

Mentioned in 1 episode

  1. 244
    The Open-Weight Defense That Feeds Attackers Confident, Falsified Answers

Related terms