Definition
Plain language
Re-running submitted code to verify it actually produces the score the paper reported.
As stated in the literature
In CoE audits, the integrity check that re-executes a paper's submitted code under the canonical evaluator to verify the reported scores; one of four checks alongside method-code alignment, citation existence, and specification compliance.
Why it matters: Without this check, a paper's headline number is just a claim — and AI-generated submissions in particular have proven prone to numbers that don't survive re-execution.
For example, the audit downloads the paper's submitted code, runs it on the official evaluator, and verifies that it really does produce the 87% accuracy the paper reported.
Heard on the show
“The reproduction check is what catches it.”Episode 089 — When AI-Written Papers Read Well But the Evidence Underneath Is Broken