Definition
Math reasoning is the cluster of capabilities involved in solving mathematical problems: symbolic manipulation, careful multi-step deduction, knowing when to fall back on computation. LLMs have made enormous progress here, much of it driven by RL on verifiable answers.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.
- Demystifying Long Chain-of-Thought Reasoning in LLMs
- Looped Transformers as Programmable Computers
- Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Model Parameters
- AlphaProof and AlphaGeometry 2: AI achieves silver-medal standard solving International Mathematical Olympiad problems
- Let's Verify Step by Step
- AIMO-2: Advancing AI Mathematical Olympiad with Open Large-Scale Training Data
- Faith and Fate: Limits of Transformers on Compositionality
- Neural Tangent Kernel: Convergence and Generalization in Neural Networks
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs
- Hypertree Proof Search for Neural Theorem Proving