Definition
Plain language
A classic decision puzzle about when to keep using what already works versus trying something new that might be better.
As stated in the literature
A reinforcement-learning framework formalizing the explore-exploit tradeoff over actions with unknown payoffs; LoopTrap's strategy selection uses bandit-style exploration weighted by behavioral priors.
Also called: bandit
Why it matters: It captures the universal explore-versus-exploit tradeoff, guiding systems that must balance using what works against discovering something better.
For example, it's like deciding each night whether to revisit your favorite restaurant or try a new one that might be even better.
Heard on the show
“That exact term is the upper-confidence-bound rule — UCB — the classic exploration bonus from bandit problems and game-tree search.”Episode 162 — The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models