Definition
Plain language
A benchmark of task-oriented conversations, like booking a hotel or restaurant, used to test dialogue systems.
As stated in the literature
A multi-domain task-oriented dialogue benchmark scored by joint goal accuracy and slot F1; used to evaluate multi-agent orchestration in GBC.
Why it matters: It matters because it provides a standard yardstick for whether a dialogue system can juggle real multi-step booking tasks.
For example, a system is tested on a conversation where a user books a hotel and then asks for a nearby restaurant, and it's scored on getting every detail right.
Heard on the show
“The benchmark is MultiWOZ — task-oriented dialogue, a manager routing to domain workers to a responder.”Episode 181 — How to Backpropagate Blame Through a Team of Chatbots — And When It Backfires