Definition
Plain language
One complete run of an agent or model attempting a task from start to finish.
As stated in the literature
A single sampled trajectory of an agent's actions and observations from initial state to terminal state, used in RL training and evaluation.
Also called: rollouts
Why it matters: Rollouts are the atomic unit of both agent evaluation and RL training — almost every metric and gradient ultimately comes from a pile of them.
For example, a single rollout of a coding agent on a bug-fix task might include reading the repo, running tests, editing three files, and submitting a final patch.
Heard on the show
“You can't have a judge read two forty-thousand-token rollouts and reliably pick the better one.”Episode 003 — How to Pick the Best of Sixteen Coding Agent Rollouts