Definition
Plain language
A small slice of a document, cut up so a search system can store and fetch pieces instead of whole pages.
As stated in the literature
The unit of text a retrieval pipeline embeds and indexes; chunk size and overlap trade off retrieval precision against context coherence, and smaller chunks break up long self-contained documents that would otherwise dominate a context window.
Also called: chunks, chunking, chunk size, chunked, unchunked
Why it matters: Chunk size decides what the model actually sees — cut too small and answers lose context, cut too large and one long document crowds out everything else.
For example, a 40-page manual might be sliced into a few hundred paragraph-sized chunks so a search system can return just the two paragraphs about resetting the device.
Heard on the show
“Each `next`, each `print`, each `step` is a full inference cycle against the model — dollars of API spend, latency, and a chunk of the context window burned for a single line of advance.”Episode 005 — Why a Debugger Designed for Humans Is the Wrong Tool for an AI Agent