Definition
Plain language
Chopping off older text when there's too much to fit.
As stated in the literature
Dropping tokens from a prompt or history to fit a context window; when done silently by the harness it confounds evaluation of a model's own memory management.
Also called: truncate, truncated
Why it matters: When the surrounding software does this without saying so, a model can be blamed for forgetting something it was never actually shown.
For example, a long conversation quietly drops its first twenty messages to make room, so the model never sees the instruction it was given at the start.
Heard on the show
“Normally that something is the plumbing — silent truncation, or somebody's retrieval stack.”Episode 245 — Fifteen Models Ran Football Clubs for Twenty Years, and Size Didn't Decide It