Definition
Plain language
A batch of automatic checks that run a program and report whether it behaved as expected.
As stated in the literature
A collection of test cases and assertions used as an acceptance criterion; authoring one requires both selecting discriminating inputs and computing correct expected outputs.
Also called: test suites
Why it matters: Test suites are increasingly used as the gate that decides whether generated code is accepted, so a flawed suite can reject working programs or wave broken ones through.
For example, a suite for a sorting function might feed it an already-sorted list, a reversed list, and a list with repeats, checking the result each time.
Heard on the show
“… can look at a program and tell you whether it's correct about nine times out of ten will write a test suite for the same problem that throws out sixty to eighty percent of independently verified correct …”Episode 233 — Why a Model Can Grade an Answer But Not Write the Answer Key