Definition
Plain language
The standard skeleton for any reinforcement-learning problem: you're in a situation, you take an action, you get a reward, repeat.
As stated in the literature
MDP — a formalism of states, actions, transitions, and rewards underlying reinforcement learning; harness-evolution work reframes scaffold editing as an MDP (configuration as state, code edit as action, benchmark score as reward) to import RL's known failure modes and defenses.
Also called: MDP
Why it matters: It gives a precise common framework for decision-making problems, so the well-studied tools and known pitfalls of reinforcement learning can be applied to new tasks.
For example, a robot in a maze sees its current spot, chooses to move left, gets a small reward for getting closer to the exit, and then faces the next spot.
Heard on the show
“So the authors reframe the whole editing process as what's called a Markov Decision Process.”Episode 147 — Agents Fail at the Body, Not the Brain: A Self-Rewriting Scaffold That Lifts a 9B Model 44 Points