Definition
Plain language
A benchmark that tests AI agents on real Android phone apps.
As stated in the literature
A mobile-device agent benchmark covering roughly twenty apps and over a hundred tasks with state-based verification of cross-app workflows.
Why it matters: Mobile interfaces are different enough from desktop browsers that they need their own benchmark, and this is a leading one for evaluating phone-using agents.
For example, an AndroidWorld task might ask an agent to find a contact in one app, copy the phone number, and paste it into a text message in another.
Heard on the show
“7 — hits that seventy-one point eight percent on the benchmark called AndroidWorld, and just under fifty-two percent on a harder one called MobileWorld.”Episode 157 — When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed