Definition
Plain language
The dominant online home for code projects and where most open-source software lives.
As stated in the literature
The widely used hosted version-control and collaboration platform for software development; many agent benchmarks draw from real GitHub issues and pull requests.
Why it matters: It's where most modern software gets built, so it's also the natural source of realistic tasks for evaluating coding agents.
For example, the SWE-Bench benchmark draws its tasks directly from real bug reports and pull requests on public GitHub repositories.
Heard on the show
“You give an AI coding agent a real bug from a real GitHub repo and tell it, fix this.”Episode 003 — How to Pick the Best of Sixteen Coding Agent Rollouts