Glossary · Term

critical token

← all terms

Definition

Plain language

The point in an AI's reasoning where its chance of success has already hit zero and stays there.

As stated in the literature

From prior work, the token after which success probability is pinned at zero; contrasted with the earlier cliff token that triggers the collapse rather than marking its aftermath.

Also called: critical tokens

Why it matters: Distinguishing the point of no return from the earlier word that caused it helps researchers locate where a fix should actually be applied.

For example, several words after a model makes a fatal arithmetic slip, it reaches a point where no continuation can possibly reach the right answer.

Heard on the show

“There's a nice contrast here with earlier work, too — the "critical tokens" idea from Lin and colleagues.”
Episode 172 — One Bad Token Can Sink a Model's Math, And You Can Delete It

Mentioned in 1 episode

  1. 172
    One Bad Token Can Sink a Model's Math, And You Can Delete It

Related terms