Definition
Needle-in-a-Haystack is a long-context evaluation method that inserts a single, often arbitrary fact into a large document and tests whether a model can retrieve it on demand. It became a standard way to advertise context-window capability, but critics argue it overstates real-world retrieval skill because the inserted “needle” is typically lexically distinct and trivially matchable, unlike genuine queries that must be disambiguated from many similar, competing passages.