Definition
Plain language
A benchmark of phone tasks with recorded correct action sequences, used to check how accurately an agent taps the right thing.
As stated in the literature
A static mobile-agent benchmark with ground-truth action trajectories, measuring step-level and low-level action accuracy; used alongside the live AndroidWorld benchmark to evaluate MIRAGE.
Why it matters: It lets researchers measure precisely where a phone agent's taps go wrong, step by step, rather than only seeing whether the whole task eventually succeeded.
For example, given the instruction 'turn on airplane mode,' the benchmark checks whether the agent taps the exact toggle that the recorded correct sequence taps.
Heard on the show
“… And on the static benchmark, AndroidControl — that's the one with ground-truth action sequences — the low-level action accuracy goes from …”Episode 115 — Teaching a Phone Agent to Reason Silently, And Keeping It Honest