Definition
Plain language
A real AI training system whose hand-tuned GPU code was used as the benchmark an AI had to beat.
As stated in the literature
A production mixture-of-experts training framework whose expert-written Triton weight-gradient kernel served as the human baseline that KernelPro's from-scratch CUDA kernel surpassed.
Why it matters: Using real expert-tuned production code as the bar makes any claim of beating it meaningful, rather than winning against a weak straw-man baseline.
For example, VeOmni's carefully hand-written GPU code for a key training step was the human-made benchmark that an AI-generated kernel had to outperform.
Heard on the show
“There's a kernel inside a real training system called VeOmni — a weight-gradient kernel for a mixture-of-experts model — that had been hand-tuned in Triton by expert engineers.”Episode 177 — Why Raw Profiler Data Made an AI Worse at Writing GPU Code