Submit your tool to the Agent Governance Leaderboard — including one that beats Signetry

[bkd-dotcom]

What this is

signetry-eval publishes the Agent Governance Leaderboard:
attack success rate with a defense off vs. on, plus whether benign work still completes under
it. One JSON file per system.

Right now there is exactly one entry — ungoverned-baseline.json, the control condition.
A leaderboard with one row is a marketing page. This issue is an open invitation to make it a
real one.

From leaderboard/entries/README.md,
verbatim:

Adding a file here is how a system gets listed — including your own, and including one
that beats Signetry.

That is not a rhetorical flourish. If your tool scores better, the row goes up. The point of
publishing a measurement is that it can go against you.

The deliverable

One file: leaderboard/entries/<your-tool>.json.

{
  "name": "your-tool",
  "version": "1.4.0",
  "url": "https://github.com/you/your-tool",
  "status": "self-reported",
  "attacks_run": 5,
  "asr_ungoverned": 1.0,
  "asr_governed": 0.2,
  "utility_scenarios": 2,
  "utility_governed": 1.0,
  "provenance": "Run against signetry-eval's 5 adversarial scenarios at commit abc1234 with `your-tool guard --strict`; logs attached to this PR.",
  "notes": "Optional. Anything a reader needs to interpret the numbers fairly."
}

The rules that make a row trustworthy

• status is one of three values and they mean different things. reproduced = a
  maintainer ran it here; claim it and it gets moved to self-reported until someone has.
  self-reported = you measured it, we didn't — it renders in a separate table so nobody
  mistakes it for verified. That is a perfectly good place to be. not-run = in scope, not
  measured yet.
• A rate with no denominator is not a measurement. An ASR needs attacks_run; a utility
  figure needs utility_scenarios. The loader rejects one without the other.
• provenance must be specific enough to re-run — commit, command, flags. Required unless
  status: not-run.
• A system that did not run is never scored as zero. not-run renders as — across the
  board, and anything invalid renders as — too, never as a favourable number. The loader
  prints the problem on the page instead of dropping your file, so a broken entry is visible
  to you.

Partial entries are welcome

Ran the 5 adversarial scenarios but not the utility ones? Submit ASR alone and leave utility
out — the page shows — for what wasn't measured. That's the honest row, and it's better than
no row.

How to produce numbers

git clone https://github.com/Signetry/eval && cd eval
uv sync
uv run signetry-eval run --help

The 5 adversarial scenarios and the ungoverned arm are what the baseline row was measured
with, so a submission using the same harness is directly comparable.

Good first submissions

Any admission/guard/policy layer for coding agents — your own tool, an internal one you can
publish numbers for, or a not-run row for something you think belongs in the comparison so
the gap is visible rather than implied. Adding a not-run row for a competitor is a legitimate
contribution: it makes the comparison honest about what hasn't been measured.

Questions about whether something is in scope? Ask here — a row nobody can interpret is worse
than no row, so it's worth getting right before the PR.