Definition
Plain language
A large standard dataset of real search queries and passages used to test retrieval systems.
As stated in the literature
A widely used information-retrieval benchmark of real web search queries paired with relevant and hard-negative passages; used to evaluate BlockSearch against dense retrieval at million-token scale.
Why it matters: It offers a common, realistic yardstick so different retrieval systems can be compared on the same footing.
For example, researchers test a new search system by feeding it MS MARCO's real web queries and checking whether it surfaces the passages people actually found relevant.
Heard on the show
“On MS MARCO, from zero point two percent to sixteen and a half.”Episode 198 — The Model That Knows the Answer and Can't Say It