Definition
Plain language
A way to peek at which internal parts of a model are pushing toward a particular output word.
As stated in the literature
An interpretability technique that projects a component's contribution through the unembedding to measure how much it moves the model's output logits; can be distorted by tied embeddings, where it spuriously peaks at layer zero.
Why it matters: It helps researchers trace where a model's decisions come from, though they have to watch for distortions like shared input-output tables that can fool the measurement.
For example, if a model outputs the word 'Paris,' this technique can show which internal pieces most strongly pushed for that word over alternatives.
Heard on the show
“There's a separate appendix where they handle a known artifact in the Gemma models, where direct logit attribution peaks at layer zero because of tied embeddings.”Episode 037 — Why Hallucination Detectors Miss Stale Facts: A Geometric Story About What Models Know But Don't Say