Definition
Plain language
A model picking up new patterns just from examples in its prompt, without any retraining.
As stated in the literature
The phenomenon where a language model adapts behavior based on examples or instructions provided in its prompt context without weight updates.
Why it matters: It's why prompt engineering works at all — users can teach a model new behavior at runtime without any access to its weights.
For example, showing a model three examples of English-to-pig-Latin translation in the prompt and then giving it a new English word, and watching it produce the right pig-Latin output.
Heard on the show
“… The conventional intuition about in-context learning is that the model is reading the demonstrations as English — picking up cues from the names …”Episode 013 — Why Search Keeps Rediscovering the Same Workflow, and What That Means