Definition
Plain language
A test that measures how much a written instruction document actually improves an AI agent's performance.
As stated in the literature
A benchmark quantifying the pass-rate lift agents gain from injected skill documents; human-written skills add roughly sixteen points while raw LLM-authored skills initially add close to nothing.
Why it matters: It quantifies how much a well-written skill document actually helps an agent, revealing that human-written guidance can help while raw machine-written guidance may not.
For example, it runs an agent on a task with and without an instruction document attached and measures how many more tasks it passes with the document.
Heard on the show
“There's a benchmark called SkillsBench that was built to measure one very specific thing.”Episode 132 — The Agent Failed — But Did the Instructions Deserve to Be Followed?