Definition
Plain language
A system that diagnoses why an AI's self-written instruction document failed and rewrites just the broken part.
As stated in the literature
An evaluation-guided self-refinement framework that runs an agent with and without a skill, diagnoses targeting, compliance, fault attribution, and coverage, and rewrites the skill from a structured improvement brief; separates 'bad instructions' from 'agent ignored good instructions.'
Why it matters: It separates bad instructions from an agent that disobeys good ones, so fixes target the real cause instead of blindly rewriting everything.
For example, when an agent's instruction file fails, SkillAxe figures out whether the instructions were wrong or the agent simply ignored them, then rewrites only the part that's broken.
Heard on the show
“The paper that digs into why — and what to do about it — is called "SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement," from a team at Microsoft.”Episode 132 — The Agent Failed — But Did the Instructions Deserve to Be Followed?