Definition
Plain language
When several of the model's ideas are wired together so tightly that changing one drags the others along.
As stated in the literature
The property that distinct concepts share overlapping directions or circuitry in a model's representation space, so an edit targeting one belief non-locally perturbs correlated beliefs; contrasts with the folk model of local, surgical output editing.
Also called: entangled
Why it matters: It means edits to a model's beliefs are rarely surgical, so a fix aimed at one behavior can quietly break unrelated ones.
For example, trying to make a model less willing to describe itself as conscious might unexpectedly also change how it talks about emotions or self-awareness.
Heard on the show
“The state stream and attention are deeply entangled — you can't ablate one without affecting the other.”Episode 032 — A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking