Definition
Plain language
A benchmark where AI agents have to complete tasks on simulated websites.
As stated in the literature
A benchmark for web-based autonomous agents, providing realistic simulated websites and multi-step tasks for end-to-end evaluation.
Also called: WebArena-Lite
Why it matters: It provides a controlled, reproducible way to evaluate web agents that doesn't break every time real websites change their layout.
For example, an agent might be asked to find the cheapest item in a category on a fake shopping site and add it to a cart.
Heard on the show
“They also evaluate on WebArena — a separate browser benchmark the model never saw during training.”Episode 080 — How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents