Definition
Plain language
When answers sneak into the study material, so a good score doesn't mean real learning.
As stated in the literature
Contamination in which information about the evaluation targets is available in training or visible test data, inflating measured performance relative to genuinely fresh data.
Also called: leakage, leaky, leaked
Why it matters: It quietly inflates scores, so a system that looks excellent in testing can fail the moment it meets genuinely new data.
For example, if a few of the exam questions were accidentally printed in the practice packet, a student's high score reflects recall rather than understanding.
Heard on the show
“The team mentions masking links to certain sites during evaluation to avoid leakage, which signals they're aware.”Episode 021 — Ten Thousand Examples Beat the Full Industrial Pipeline for Search Agents