Definition
Plain language
Training examples generated by AI rather than collected from the real world.
As stated in the literature
Data produced by simulation or by other models — often used to scale training in agentic, math, and code domains where verifiable rewards or rubric trees can be constructed.
Why it matters: It lets training scale past what humans can label, especially in domains like math and code where correctness can be checked mechanically.
For example, a math model might be trained on millions of equations generated and graded automatically, rather than scraped from textbooks.
Heard on the show
“Most synthetic data work treats validation as the last stage — generate, then filter.”Episode 059 — Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward