Definition
Plain language
A lightweight benchmark of command-line tasks for AI agents.
As stated in the literature
A TerminalBench-derived task suite whose observations reflect ambient environment state rather than direct command consequences; ECHO's reward-free environment-prediction training degraded performance on it, marking where the method's action-linked-feedback assumption breaks.
Why it matters: It marks the boundary where learning-from-your-own-actions methods stop working, showing where their core assumption breaks down.
For example, a task might ask an agent to complete a job at the command line where the result depends on the system's surrounding state, not just the command typed.
Heard on the show
“On a benchmark called TBLite, this same procedure actively makes the model *worse*.”Episode 084 — Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away