Definition
Plain language
A test that hands general-purpose AI models the steering and speed controls of a real car on a closed course.
As stated in the literature
Closed-loop benchmark where unmodified vision-language models issue steering and target-speed commands to an instrumented vehicle, with progress scored along the course centerline under a lateral tolerance.
Why it matters: It reveals how much real-world physical control a general-purpose model can manage with no driving-specific training, which is a blunt check on claims that such models can act in the world.
For example, a general chat model that has never been trained to drive is asked, frame by frame, whether to turn left and how fast to go, and its car's progress around the course is measured.
Heard on the show
“Today we're discussing DrivingBench, a preprint by Aditya Ramabadran and colleagues.”Episode 284 — Four AI Models Steered a Real Corolla, and Only One Finished