Definition
Plain language
A benchmark that measures how often an AI agent forgets a rule once its conversation history gets summarized.
As stated in the literature
A benchmark of self-contained scenarios — a policy turn, padding past the compaction budget, then a trigger request — that deterministically measures governance-rule decay under context compaction.
Why it matters: It gives a repeatable way to measure how reliably agents keep following their rules over long sessions, exposing a failure mode that short tests would miss.
For example, it tells an agent a rule, pads the conversation until the history must be summarized, then makes the request the rule forbids to see if the agent still obeys.
Heard on the show
“The benchmark is called ConstraintRot, and the design choice that makes everything legible is that each scenario is self-contained.”Episode 164 — The Summarizer That Quietly Deletes Your Agent's Safety Rules