Definition
Test-time compute is the amount of computation a model spends per query at inference — sampling more candidates, running longer chains-of-thought, searching deeper. Trading inference compute for capability has been one of the biggest stories of the last couple of years.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters
- Universal Transformers
- TTRL: Test-Time Reinforcement Learning
- Let's Verify Step by Step
- Hypertree Proof Search for Neural Theorem Proving
- Let's Think Step by Step: Large Language Models are Zero-Shot Reasoners
- Training Large Language Models to Reason in a Continuous Latent Space (Coconut)
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach