Definition
Plain language
A test set of Python scripting tasks for AI agents working at a command line.
As stated in the literature
A Python-scripting benchmark for terminal agents where command outputs are tightly coupled to the agent's own actions; used in ECHO's verifier-free experiments, where reward-free environment-prediction training improved task success.
Why it matters: Because each command's result depends directly on the agent's own actions, it serves as a clean testbed for whether agents can learn from predicting what their commands will do.
For example, a task might ask an agent to write and run a short Python script at the command line and check that its output is correct.
Heard on the show
“On a Python scripting benchmark called PyTerm, pass rate goes up by ten percentage points.”Episode 084 — Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away