Concept · 3 episode(s)

Token-Level Analysis

← all concepts

Definition

Token-level analysis studies a model’s behavior one token at a time — logits, top-k, entropy, attention — rather than looking only at the final answer. It’s the natural granularity for many interpretability and decoding questions.

Episodes covering this

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.