Definition
Plain language
A benchmark of trivia questions that require pulling facts from multiple Wikipedia pages.
As stated in the literature
A multi-hop question-answering dataset over Wikipedia requiring cross-document reasoning to combine facts from multiple supporting paragraphs.
Why it matters: Multi-hop questions force systems to actually chain retrieval and reasoning rather than rely on a single lucky lookup, which is closer to real research tasks.
For example, a HotPotQA question might ask 'What year did the director of Inception graduate from college?' which requires looking up both the director and then their biography.
Heard on the show
“And HotPotQA — the multi-hop benchmark this paper uses — happens to have two flavors of question that map exactly onto this distinction.”Episode 011 — When RL Actually Teaches Agents Something New, And When It Doesn't