Definition
Plain language
A benchmark for testing whether an AI search agent picks the right web source.
As stated in the literature
An evaluation suite for generative-search source selection under GEO-poisoned and clean conditions, used to probe persuasion susceptibility in retrieval agents.
Why it matters: It measures how easily generative search agents can be steered by adversarially crafted pages, which is a live security concern as these systems get deployed.
For example, Geo-Bench might present a search agent with a question and two web pages — one neutral, one stuffed with persuasion text — and check which one the agent cites.
Heard on the show
“They built a second benchmark called Geo-Bench, designed to be more realistic.”Episode 038 — How LLMs Get Persuaded: One Attention Head, A Tetrahedron, And A Single Dial