Definition
Plain language
Adding extra small rewards along the way to guide a learning agent toward a goal that's otherwise rarely reached.
As stated in the literature
Augmenting a sparse reward with auxiliary intermediate signals to ease credit assignment; potential-based shaping (Ng et al.) provably preserves the optimal policy while accelerating learning.
Also called: shaping reward, potential-based shaping
Why it matters: It makes learning feasible on tasks where success is rare, and a careful form of it can speed learning without changing the best final behavior.
For example, instead of rewarding a robot only when it reaches the exit, you give it small rewards for getting closer to guide it along the way.
Heard on the show
“They apply the shaping reward only to the correct samples — stripping out the partial-progress signal entirely — and they still get a twenty-four point improvement on hard Countdown.”Episode 081 — When Reasoning Models Decide Before They Think: Detecting and Fixing Premature Confidence