Definition
Plain language
Knowledge baked permanently into a model during training, as opposed to facts it's reading right now in the conversation.
As stated in the literature
The regime where information is encoded in a model's parameters via training, contrasted with in-context information held transiently in the prompt; the two can yield opposite conclusions from identical text, as in negation neglect.
Also called: in-weights belief, in-weights learning
Why it matters: Distinguishing baked-in knowledge from what's in the prompt matters because the two can clash and lead a model to opposite conclusions from the same text.
For example, a model 'knows' the capital of France from its training, separate from a fact you just typed into the chat a moment ago.
Heard on the show
“And that's the conceptual hinge of the whole paper — the gap between in-context and in-weights.”Episode 043 — When 'This Is False' Doesn't Stick: Why Models Learn the Lie Anyway