Definition
Plain language
A setting where, after you act, something random and outside your control also happens.
As stated in the literature
An environment with non-deterministic transitions where the next state isn't fully determined by the agent's action; the leftover value terms it introduces break the telescoping reward-recovery and motivate advantage-based signals.
Also called: stochastic, deterministic environment
Why it matters: Randomness in outcomes makes it hard to tell whether a result came from a good decision or luck, which is exactly why some training methods focus on advantage rather than raw reward.
For example, in a card game, after you choose to draw a card, which card you actually get is random and beyond your control.
Heard on the show
“To be clear about scope — this shifts the odds in a stochastic, million-dimensional landscape; it doesn't lock in the destination.”Episode 199 — Finding a Model's Hidden Behaviors Without Knowing What You're Looking For