Definition
Plain language
A rule for exploring options that gives a little extra benefit of the doubt to choices you've tried less often.
As stated in the literature
UCB — the classic exploration heuristic from bandit problems and tree search that adds an optimism bonus inversely related to how often an option has been sampled; in InfoTree this term falls out directly from the marginal gain of the informativeness objective.
Also called: UCB
Why it matters: It balances trying new options against sticking with known good ones, which is key to making smart choices when you can't test everything.
For example, when choosing which restaurant to try, you'd lean toward a promising one you've barely visited because its true quality is still uncertain and might be great.
Heard on the show
“That exact term is the upper-confidence-bound rule — UCB — the classic exploration bonus from bandit problems and game-tree search.”Episode 162 — The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models