Definition
Plain language
The optimizer's own private notebook of lessons about how to do its job well.
As stated in the literature
In SkillOpt, a persistent training-time document the optimizer writes for itself, containing craft heuristics about which kinds of edits tend to help; invisible to the student model.
Why it matters: Separating the optimizer's lessons from the student's content keeps the student document focused while letting the search process compound its own learning.
For example, the optimizer's private notebook might accumulate the heuristic 'when adding a worked example, also add a counter-example,' which it consults before each edit but the student never sees.
Heard on the show
“So the question the authors pose is: can you train a model end-to-end, with reinforcement learning, to have a meta-skill?”Episode 160 — Training an AI to Take Its Own Notes, So Its Future Self Works Better