epoch

Definition

Plain language

One full pass through a dataset during training.

Also called: epochs

Why it matters: It's a natural unit for scheduling training events like learning-rate drops, checkpointing, and consolidation steps, and it's a quick proxy for how much the model has seen.

For example, if you train on a million examples and finish looking at every one of them once, you've completed one epoch.

Heard on the show

“5 for a hundred and thirty optimizer steps — one epoch over a twelve-thousand-sample subset of FineVision, image-question pairs with no answers attached.”

Mentioned in 12 episodes

1. 242
   Making a Vision Model Better by Showing It Blurry Images
2. 237
   The Model Built a Perfect Map of the Puzzle, Then Lost It
3. 221
   Two Hundred Clean Economics Answers, And a Model That Endorses Race Science
4. 220
   Write Like It's 1923: The One-Prompt Trick That Beats AI Detectors
5. 212
   The Fact Was in the Wrong Drawer: Why Fine-Tuned Models Can't Reason With What They Know
6. 178
   How an AI Reviewer Learned to Stop Going Easy on AI Writing
7. 156
   Why More Human Demonstrations Made a Computer-Use Agent Worse
8. 085
   Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction
9. 078
   Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
10. 059
    Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward
11. 054
    When Models Learn the Monitor Exists, the Reasoning Trace Stops Being a Window
12. 026
    What RL Actually Does to Language Models, at the Token Level

Related terms