Definition
Plain language
A dispatcher that reads your question and decides which AI model should answer it.
As stated in the literature
Pre-generation model-selection system optimizing accuracy against per-token cost; evaluated against a cost-matched weighted-random baseline over a fixed model pool.
Also called: LLM Routers, LLM routers, LLM Router, large language model router
Why it matters: Done well it cuts costs sharply without hurting answers, but a badly designed one quietly sends your hardest questions to your weakest model.
For example, a router might send "what's the capital of Peru" to a tiny cheap model and "review this contract for liability risks" to the most capable one.
Heard on the show
“That's the central tension in "Dynamic LLM Routers are Often Misguided," a preprint from Sam Wang and colleagues at Fastino Labs.”Episode 286 — Six LLM Routers Tested, And None Beat a Weighted Coin Flip