Definition
Plain language
A baseline approach for using a strong LLM as a safety judge over proposed agent actions.
As stated in the literature
A schema-aware LLM-as-judge safety architecture for agentic systems, used as the primary neural baseline in evidence-carrying agent comparisons.
Why it matters: LLM-as-judge safety layers are a popular pattern, and benchmarking against them shows whether new verifier designs are actually better.
For example, before an agent runs a shell command, Progent asks a strong LLM judge to check whether the command matches a safe schema.
Heard on the show
“They run a state-of-the-art frontier model as a Progent-style safety judge — schema-aware, gets to inspect the action and the context, gets to reason about whether it's safe.”Episode 062 — Treating Hallucinations as Exploits: A Gate-Based Architecture for Agent Safety