Definition
Plain language
A two-player situation where one player commits to a move first and the other responds optimally.
As stated in the literature
A leader-follower game in which one player (the leader) commits to a strategy first, and the other (the follower) best-responds; used to model the RLHF policy-and-reward-model dynamic.
Also called: Stackelberg
Why it matters: Many AI training setups — like RLHF, where the reward model commits and the policy responds — have exactly this leader-follower structure, and the game-theoretic view sometimes catches failure modes the standard view misses.
For example, a chain restaurant publicly announces its prices first, and a small competitor sets its own prices afterwards, knowing what the chain will charge.
Heard on the show
“And without strong convexity, the inverse Hessian is undefined, the implicit function theorem doesn't apply, and the unique best response that the Stackelberg framing depends on doesn't exist.”Episode 025 — The Missing Gradient Term That Predicts Sycophancy in RLHF