Definition
Plain language
An improved tool for peeking at what a model is computing layer by layer.
As stated in the literature
A trained variant of the logit lens that learns per-layer projections to the unembedding, giving cleaner intermediate-prediction read-outs than the raw projection.
Why it matters: Cleaner per-layer read-outs make it possible to actually trace where in the model a decision is formed, which the raw logit lens often obscures.
For example, a tuned lens can show that by layer 20 a model has essentially decided on its answer, with later layers just refining the phrasing.
Heard on the show
“There's a more recent variant, the tuned lens, that addresses this.”Episode 018 — Language Models Compute the Rational Move, Then Override It