Concept · 5 episode(s)

Logit Lens

Definition

The logit lens reads the model’s output unembedding off intermediate layers to see what the next-token distribution looks like as computation progresses. It’s a quick way to ask “at what layer does the model already know the answer?”

Episodes covering this

203
The Thought a Model Doesn't Say — and the Lens That Reads It
Verbalizable Representations Form a Global Workspace in Language Models
Gurnee, Sofroniew, Pearce et al. · Anthropic·16 min·Jul 07, 2026
141
How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
Yang, Chen, Wu et al. · HKUST(GZ)·29 min·Jun 12, 2026
094
Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most
The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
Onyame, Zhou, Thopalli et al. · University of Virginia·24 min·May 28, 2026
074
How a Fifteen-Hundred-Dollar Training Run Matched Llama and Gemma on Reasoning
HRM-Text: Efficient Pretraining Beyond Scaling
Wang, Liu, Wang et al. · Sapient Intelligence·21 min·May 24, 2026
018
Language Models Compute the Rational Move, Then Override It
What Suppresses Nash Equilibrium Play in Large Language Models? Mechanistic Evidence and Causal Control
Lekeas, Stamatopoulos · DreamWorks Animation·29 min·May 03, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.

Eliciting Latent Predictions from Transformers with the Tuned Lens