Definition
Plain language
A model picking up new patterns just from examples in its prompt, without any retraining.
As stated in the literature
The phenomenon where a language model adapts behavior based on examples or instructions provided in its prompt context without weight updates.
Why it matters: It's why prompt engineering works at all — users can teach a model new behavior at runtime without any access to its weights.
For example, showing a model three examples of English-to-pig-Latin translation in the prompt and then giving it a new English word, and watching it produce the right pig-Latin output.
Heard on the show
“The hand-note is in-context learning.”Episode 160 — Training an AI to Take Its Own Notes, So Its Future Self Works Better