Definition
Plain language
The knack for picking which experiments are worth running and reading what their results really mean.
As stated in the literature
Experimental judgment measured separately from implementation ability: selecting informative experiments, interpreting noisy outcomes, and deciding when to stop or iterate.
Also called: experimental research taste
Why it matters: Most of the cost in research is spent on experiments that teach you nothing, so judgment about which ones to run determines how fast progress happens.
For example, two people can both write flawless code, but only one of them notices that the promising-looking result is just noise and that the next experiment should change the data, not the model.
Heard on the show
“Their target is experimental research taste: deciding which experiments to run, and interpreting what happens.”Episode 287 — Can You Measure Research Taste If The AI Isn't Allowed To Code?