Definition
Plain language
A simulated online shopping site used to test whether AI agents can find and buy the right product.
As stated in the literature
A benchmark environment in which an agent must navigate a mock e-commerce site to locate and purchase a product matching natural-language criteria, widely used to evaluate web and tool-use agents.
Why it matters: It offers a repeatable testbed for measuring whether AI agents can actually carry out real-world web tasks like shopping.
For example, an agent might be told to 'buy a red cotton t-shirt under twenty dollars' and graded on whether it navigates the mock store and picks the right item.
Heard on the show
“… During training the typical shared node pools about five attempts, and only around twelve percent of WebShop steps — eight percent on the household tasks — are singletons that nobody else ever revisited. …”Episode 165 — A Free-Lunch Tweak That Lets a Tiny Agent Beat Frontier Giants