Definition
Iterative refinement has a model improve its output across multiple passes, often using its own critique or an external verifier as feedback. It often helps; it sometimes hurts (the model can talk itself out of correct answers); the wins are tied to whether the critique signal is reliable.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.
- Self-Refine: Iterative Refinement with Self-Feedback
- Universal Transformers
- Deep Equilibrium Models
- Looped Transformers as Programmable Computers
- FunSearch: Making new discoveries in mathematics using large language models
- OpenEvolve: Open-Source Implementation of AlphaEvolve
- FunSearch: Making new discoveries in mathematical sciences using large language models
- FunSearch: Making New Discoveries in Mathematical Sciences Using Large Language Models
- Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs
- Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Model Parameters
- TextGrad: Automatic 'Differentiation' via Text
- Reflexion: Language Agents with Verbal Reinforcement Learning