Definition
Plain language
A way to make a sealed, API-only AI agent better at a task without ever changing its weights — by running many copies, scoring them, and keeping the good ones.
As stated in the literature
A gradient-free method exploiting the RL-as-inference equivalence to sample from the reward-tilted posterior over a frozen black-box agent's trajectories via Sequential Monte Carlo, using a small offline-trained value model as the resampling critic.
Also called: AMC
Why it matters: It lets people improve AI agents they can only access through an API, without the access or cost of retraining the underlying model.
For example, you could take a customer-service bot you can only reach through an API, run many copies on the same request, score each conversation, and keep the best ones to lift its performance.
Heard on the show
“The paper is called "Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents.”Episode 119 — Beating Reinforcement Learning Without Ever Touching the Model's Weights