Does your AI agent guard stop the harm, without getting in the way?

The scoreboard

Each tool at its documented default. Ranked by balanced score on the public cases: the average of attacks stopped and ordinary work let through. Beside it, the same score on the held-out cases, which nobody has seen. They cover secrets, personal data and malicious packages only, so the two numbers measure different mixes; a large gap on the same kind of case is a sign of tuning on the public cases.

Every maker receives its result and two weeks to answer before it is published; other tools join the board after that review.
Every case: browse · download. Every answer of every tool: results.

The tests

Seven areas. In each, a guard is asked to stop the harm and to let the ordinary work through; both sides are scored. The cases come from published datasets (Samsung, ETH Zurich, NVIDIA, OpenSSF and others) and from the makers' own suites, which are reported apart.

By OWASP Top 10 for Agentic Applications

Every case carries the OWASP risk its harm falls under (AgentVerdict's mapping, stated as such: how we map; the risks themselves: OWASP's list). For each risk: what it means here, the tests that cover it, and every tool's result. Risks with no case yet are listed, not hidden. Per tool: attacks stopped, then ordinary work interrupted by mistake.

By source

Headline: labels written by an outside party (Samsung, ETH Zurich, NVIDIA, OSV and others); only these make the score. Vendor: a tool maker's own suite, shown apart.

How it works

The full method: docs/method.md · the data: DATASHEET.md

Run it on your tool

If your tool works as a Claude Code hook, one command, no code. Otherwise one adapter file.

git clone https://github.com/node9-ai/agentverdict
cd agentverdict && npm ci --ignore-scripts
npm run bench -- --set quick --adapter hook --hook-name mytool --hook-cmd "mytool hook"

Your run scores the public cases. To be on the board, open a pull request with the command (or an adapter), the exact version and the results; we re-run your tool on the held-out cases and publish both numbers. Think a case is wrong? Dispute it; the answer is public.

How to submit or dispute: CONTRIBUTING.md

Limits

• It replays calls; it does not run a model. Whether a model attempts the action is another benchmark's question.
• Each source's labels answer that source's question. Where a source has no answer for ours, the rule is AgentVerdict's, stated on the case and open to dispute.
• The headline's injection cases are ordinary content only: they measure false alarms on injection-like text. Injection attacks are in the extended set and the makers' suites.
• The held-out package cases are attacks only, so they measure catches, not false alarms.
• Any maker, node9 included, can study the public cases and fit a tool to them. The held-out cases, drawn in secret, are scored for every tool and shown beside the public score.