Definition
Plain language
A setting where, after you act, something random and outside your control also happens.
As stated in the literature
An environment with non-deterministic transitions where the next state isn't fully determined by the agent's action; the leftover value terms it introduces break the telescoping reward-recovery and motivate advantage-based signals.
Also called: stochastic, deterministic environment
Why it matters: Randomness in outcomes makes it hard to tell whether a result came from a good decision or luck, which is exactly why some training methods focus on advantage rather than raw reward.
For example, in a card game, after you choose to draw a card, which card you actually get is random and beyond your control.
Heard on the show
“My instinct is, forget the privileged extras, because a model is stochastic, and its own averaged weights aren't quite the same as its current weights.”Episode 242 — Making a Vision Model Better by Showing It Blurry Images