Definition
Plain language
A batch of automatic checks that run a program and report whether it behaved as expected.
As stated in the literature
A collection of test cases and assertions used as an acceptance criterion; authoring one requires both selecting discriminating inputs and computing correct expected outputs.
Also called: test suites
Why it matters: Test suites are increasingly used as the gate that decides whether generated code is accepted, so a flawed suite can reject working programs or wave broken ones through.
For example, a suite for a sorting function might feed it an already-sorted list, a reversed list, and a list with repeats, checking the result each time.
Heard on the show
“Both are English-language coding tasks, both have discrete pass-fail outcomes from automated test suites, both fit on a single machine.”Episode 003 — How to Pick the Best of Sixteen Coding Agent Rollouts