Definition
Plain language
A small attention-based add-on that pokes at a model's internal state with a set of learned questions to pull out specific information.
As stated in the literature
A lightweight querying transformer module that probes a representation with learnable query tokens; in MIRAGE it reads the latent reasoning slots and predicts next-screen features during training, then is discarded at inference.
Why it matters: It provides a compact way to pull targeted information out of a model's internals during training, then can be removed so it adds no cost at run time.
For example, it sends a set of learned 'questions' into the model's hidden state to extract just the details needed to predict the next phone screen.
Heard on the show
“They bolt on a second little head — built from something called a Q-Former, which is just a lightweight attention module that probes a representation with learnable questions.”Episode 115 — Teaching a Phone Agent to Reason Silently, And Keeping It Honest