Definition
Plain language
How flexible a model still is to improve on its own after early training, before it gets locked into rigid habits.
As stated in the literature
The proposed property that heavy imitation pretraining freezes a policy into copying demonstrated behavior, reducing its capacity to improve through subsequent RL; offered as the reason a lighter warm start outperformed a heavier one in OpenWebRL.
Why it matters: It matters because it explains why a lighter initial training can leave a model more able to keep improving than a heavily pre-trained one.
For example, a model drilled extensively on copying expert moves may become so set in those habits that later trial-and-error practice barely improves it.
Heard on the show
“Their explanation is policy plasticity.”Episode 111 — How a 4B Web Agent Beat Models 60x Its Size on 500 Demonstrations