learning rate schedule

Definition

A plan for how big each training step should be over time, usually shrinking as training goes on.

A function controlling the optimizer step size during training, often cosine or warmup-decay; analogized in SkillOpt as a bounded-edit-count schedule for skill document updates.

Also called: learning rate

Mentioned in 2 episodes

1. 078
   Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
2. 053
   An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script

Related concepts