Definition
Plain language
An AI that can take in both pictures and text and reason about them together.
As stated in the literature
A model trained over paired image and text inputs that produces text outputs; the backbone for GUI and computer-use agents that read screenshots, and for unified multimodal generation systems.
Also called: vision-language models, VLM, VLMs
Why it matters: It lets AI handle tasks that mix pictures and words, like reading a screenshot or describing an image, which text-only systems simply cannot do.
For example, you can show such a model a photo of your open refrigerator and ask what meals you could make from what's inside.
Heard on the show
“One single vision-language model plays every role — it generates the goal, it judges feasibility, and it executes the task.”Episode 156 — Why More Human Demonstrations Made a Computer-Use Agent Worse