Definition
Plain language
A special hidden marker a language model emits to signal it has finished its answer.
As stated in the literature
The end-of-sequence token whose generation terminates autoregressive decoding; there is no explicit length variable, so output length emerges from when this token is sampled.
Also called: stop tokens, end-of-sequence token
Why it matters: It's what lets a model decide on its own when an answer is complete, since there's no separate setting that fixes the length in advance.
For example, after finishing a sentence, the model emits a hidden marker that tells the system to stop generating rather than keep rambling.
Heard on the show
“" The model just keeps going until it happens to emit a special stop token.”Episode 204 — The Length Estimate Hiding Inside a Word-by-Word Model