Definition
Plain language
A pipeline that builds verified AI-agent training data by running tools first and writing the question afterward.
As stated in the literature
A synthetic data system that explores real APIs via a tool compatibility graph, records execution graphs, and back-chains tasks whose ground-truth answers are extracted from observed outputs.
Why it matters: Working backward from real tool executions gives training data whose ground-truth answers are guaranteed to be reachable, avoiding hallucinated benchmarks.
For example, instead of asking a labeler to invent a question about a weather API, Firefly calls the API, observes the results, and then writes the question that those results answer.
Heard on the show
“It's called "Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs," it went up on arXiv on May seventeenth, twenty-twenty-six, and we're recording three days later.”Episode 059 — Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward