Definition
Plain language
Doing expensive AI work in advance, when no user is waiting, so the live response stays fast.
As stated in the literature
A family of techniques (including the contemporary Lin et al. work and the "Language Models Need Sleep" paper) that perform offline pre-processing — pre-generating likely queries or running depth-recurrent consolidation — between user requests to reduce inference latency.
Also called: sleep
Why it matters: It moves heavy thinking off the user-facing critical path, which can dramatically cut perceived latency without sacrificing answer quality.
For example, overnight a model pre-computes likely follow-up answers for tomorrow's users so morning queries return in milliseconds.
Heard on the show
“A rubric like "the answer should contain the value one hundred forty-four" is something a one-point-seven-B model can pattern-match in its sleep.”Episode 019 — When the Best Reward Model Trains the Worst Policy: Inside EvoLM