Definition
Plain language
A kind of data or input — text, images, audio, or video — that a model can work with.
As stated in the literature
A distinct type of data stream (text, image, audio, video, action); models are called multimodal when they operate over more than one, and agent frameworks route each modality to a specialist that can handle it.
Also called: modalities
Why it matters: It defines what kinds of input a system can understand, so supporting more modalities lets an AI work with the world the way people do — through sight, sound, and words together.
For example, a photo is one modality and its spoken caption is another, and a model that can handle both is called multimodal.
Heard on the show
“He treats the modality — the form the problem is presented in — as the axis of diversity.”Episode 191 — How One Researcher Beat GPT-5.2 and Gemini 3 by Judging Their Answers, Not Improving Them