Definition
Plain language
A test of how well an AI agent can answer questions by searching and reading across the web.
As stated in the literature
A web-research question-answering benchmark used as a held-out transfer evaluation for search agents, e.g. measuring whether harness improvements learned on one benchmark generalize.
Why it matters: It tests whether improvements made to a search agent on one task actually carry over to fresh, unseen web-research questions.
For example, an agent might be asked a question whose answer requires finding and reading several different web pages, and it is scored on whether it gets the answer right.
Heard on the show
“On DeepSearchQA, from sixty-one percent to sixty-nine.”Episode 131 — Why Autonomous Research Agents Forget Their Own Lessons, and Arbor's Fix