Definition
Plain language
A benchmark for testing AI agents on real desktop applications.
As stated in the literature
A computer-use benchmark covering nine real software applications and a few hundred cross-application tasks with state-based verification.
Also called: OSWorld-MCP, OSWorld-Verified
Why it matters: It's one of the few benchmarks that tests AI agents on real desktop software with state-based grading, much closer to actual computer use.
For example, a task might ask the agent to open a spreadsheet, sort a column, and paste the result into an email draft.
Heard on the show
“The standard desktop benchmark for computer-use agents is called OSWorld.”Episode 017 — When the Agent Grades Its Own Homework: A Brutal New Benchmark for AI Workers