Definition
Plain language
A plan for how big each training step should be over time, usually shrinking as training goes on.
As stated in the literature
A function controlling the optimizer step size during training, often cosine or warmup-decay; analogized in SkillOpt as a bounded-edit-count schedule for skill document updates.
Why it matters: A well-chosen schedule can be the difference between a model converging cleanly and one that either stalls or oscillates.
For example, a cosine schedule starts the learning rate high to make fast progress, then smoothly decays it to a small value so the model can settle into a good solution.
Heard on the show
“The agent can change anything: architecture, optimizer, learning rate schedule, batch size — anything except the data pipeline and the evaluation harness.”Episode 053 — An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script