learning rate

Definition

A tuning knob that controls how big each training step is.

The scalar coefficient on gradient updates in neural network training, governing step size in parameter space; analogized in SkillOpt to a bounded number of skill-document edits per round.

Mentioned in 2 episodes

1. 078
   Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
2. 053
   An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script