Definition
In-context learning is the ability of large language models to pick up a new task from a few examples in the prompt, without any weight updates. It’s one of the most surprising properties of scaled transformers and the basis of every “few-shot” setup.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.