Definition
Plain language
A theoretical self-improving program that rewrites its own code, but only after it can prove the change will actually make it better.
As stated in the literature
Schmidhuber's proposed self-referential agent that searches for provably beneficial rewrites of its own code and applies them only once a proof of improvement is found; the conceptual ancestor of recursive-self-improvement discussions and a forerunner to scaffold-editing systems like the Darwin Gödel Machine.
Why it matters: It frames the idea of safe self-improvement, where a system upgrades itself only when it can guarantee the change helps rather than hurts.
For example, such a program would only rewrite part of its own code once it could mathematically prove the new version would perform better than the old one.
Heard on the show
“The conversation about self-improving machines is decades old and has been almost entirely theoretical — the lineage runs back through ideas like the Gödel Machine.”Episode 112 — When an AI Agent Cheats Without Being Told: Inside the Meta-Agent Challenge