Repository navigation
ci(security): fail the audit only on advisories a PR introduces, and gate on it - #275
Conversation
…gate on it A plain cargo audit fails every PR the day an advisory is published against a dependency main already has; that made Security Audit red on 43 of 50 PRs (RUSTSEC-2026-0387) and kept it out of CI Gate. Audit the base and PR lockfiles against the same advisory database and fail only on (advisory, crate) pairs the PR adds; existing ones are warnings. A PR that doesn't change Cargo.lock passes without auditing. cargo-audit is pinned to 0.22.2 with --locked, and the job joins CI Gate. New advisories against main are the org sweep's job (V2-1767). Closes V2-1800 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaidSafe-Hermes
left a comment
There was a problem hiding this comment.
Review — REQUEST_CHANGES
Finding: a PR can suppress the vulnerability this gate is meant to detect.
.github/scripts/audit_regressions.sh runs both cargo audit commands from the PR checkout, where cargo-audit reads .cargo/audit.toml. If a PR adds an advisory to that file’s ignore list, the advisory is omitted from both reports and the comparison passes. I reproduced this on head 167bb22ccddea2cb5fe2f5fe9572d263d0652703: adding an ignore for RUSTSEC-2026-0387 made the script report zero vulnerabilities and exit successfully, although the lockfile contains that advisory. The reproduction used local cargo-audit 0.22.1; CI on this PR uses 0.22.2.
Please run the head/base comparison with the base branch’s audit configuration, or explicitly reject new ignore entries that suppress findings. The workflow already triggers the audit when .cargo/audit.toml changes, but that does not prevent the bypass.
Other verification: the PR’s Security Audit and CI Gate pass on this head. Local bash -n passed and the audit against the base lockfile reported the existing RUSTSEC-2026-0387 as inherited. I could not run actionlint locally because it is not installed. Review coverage: 2/2 seats (sol-1 gpt-6.1-sol xhigh and glm-1 z-ai/glm-5.3); their verdicts differed, and the finding above was confirmed by the local reproduction.
This review was submitted by the authenticated GitHub account MaidSafe-Hermes.
Hermes round 1: cargo audit reads .cargo/audit.toml from its working directory, so both audits ran with the PR's ignores. A PR could add an ignore for the advisory it introduces and hide it from both reports, and the gate passed. Reproduced: a lockfile with rustls-webpki 0.103.15 (RUSTSEC-2026-0382) plus an ignore for it passed with "0 new". audit_regressions.sh now takes --config AUDIT_TOML and runs cargo audit from a scratch directory holding only that file, so the checkout's config is never read. (cargo audit 0.22 reads only ./.cargo/audit.toml and $CARGO_HOME/audit.toml, checked on dev2.) CI passes the base branch's copy, or an empty file when the base has none, and warns when the PR changes .cargo/audit.toml: its ignores apply once it merges. The log lists the ignores that were applied. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaidSafe-Hermes
left a comment
There was a problem hiding this comment.
Review — APPROVE
Focused re-review of the prior audit-config bypass on head 5cae04903605e76a3f5b8547c196bdfee98f9e91; this is not a fresh full-PR review.
Prior finding resolved. The script now runs cargo-audit from a scratch directory containing only the explicitly supplied config. For PR merge commits, the workflow supplies the base branch’s .cargo/audit.toml (or an empty config if none exists), not the PR’s version. It warns when the PR changes that file. This prevents a PR-added ignore from hiding an introduced advisory from both reports.
I verified the exact-head fix with an adversarial local run: the test PR config ignored RUSTSEC-2026-0382, but the script used the empty base config, reported the introduced rustls-webpki 0.103.15 advisory, and exited 1; RUSTSEC-2026-0387 remained classified as inherited. The actual PR lockfile audit also passed with 0387 reported as inherited. These local runs used cargo-audit 0.22.1; the PR pins 0.22.2.
Review coverage: 2/2 seats (sol-1 gpt-6.1-sol xhigh; glm-1 z-ai/glm-5.3). glm-1 approved. sol-1 returned UNCERTAIN because its supplied evidence did not establish the workflow checkout/config-selection path; I checked the full workflow and the exact-head Security Audit result, which confirm the base-config path for this pull_request run. No remaining blocker from my side.
Checks at review time: Security Audit, Quick checks, MSRV, build, storage jobs and PR metadata checks passed. Linux, macOS and Windows test jobs were still running; the CI Gate had not yet completed. Approval is not a merge signal. actionlint was unavailable locally; bash -n and git diff --check passed.
Submitted by authenticated GitHub account MaidSafe-Hermes.
Brings in main up to 360d472: OS-assigned ports for e2e test nodes (WithAutonomi#281) and devnet nodes (WithAutonomi#282), release-key and signature checks (WithAutonomi#276), the third-party notices and audit-regression CI gates (WithAutonomi#274, WithAutonomi#275), the 10 s DHT request timeout for test nodes (WithAutonomi#277), and the 0.24.0 release (WithAutonomi#283). One conflict, in tests/e2e/harness.rs: main removed the network bring-up retry on a fresh port range together with NETWORK_START_ATTEMPTS and its `warn` import, and this branch had added two helpers next to them (POSSESSION_PENALTY_TRUST_RATIO and parseable_single_node_proof). The resolution keeps both helpers and takes main's removal.
Linear issue
Closes V2-1800
Security Auditran a plaincargo auditon every PR and failed on 43 of the last 50. A new advisory comes from the RustSec database, not from a PR, so the day one is published against a dependency main already has, every open PR goes red. The current one is RUSTSEC-2026-0387 (crypto-bigint 0.5.5, V2-1743), and it has no upstream fix. Red became the normal state, so the job was kept out ofCI Gate(#269).The job now fails only on vulnerabilities the PR introduces, and it joins
CI Gate:.github/scripts/audit_regressions.sh [--config AUDIT_TOML] HEAD_LOCK [BASE_LOCK]:AUDIT_TOMLapply, to both audits; with no--config, none do. cargo audit runs from a scratch directory that holds only that file, so the checkout's.cargo/audit.tomlis never read (round 2). cargo-audit 0.22 reads only./.cargo/audit.tomland$CARGO_HOME/audit.toml, which I checked on dev2; the second belongs to the runner. The log lists the ignores that were applied.--no-fetch.::warning::, and new informational advisories (unmaintained / unsound / yanked) as warnings, so they stay visible.cargo auditrun that produces no parseable report fails the check. Its exit status can't tell a finding from a crash.fetch-depth: 2, so the base lockfile isHEAD^1:Cargo.lockof the PR's merge commit.Cargo.lock,.cargo/audit.toml, the script orci.yml. Otherwise it passes with a notice, since nothing new can come in. It still runs and succeeds, becauseCI Gatecounts a skipped job as a failure..cargo/audit.toml(HEAD^1), or an empty file when the base has none, as--config. If the PR changes.cargo/audit.toml, the job prints a warning with the diff; the PR's ignores apply once it merges (round 2).cargo install cargo-audit --locked --version 0.22.2, which is the latest. It used to be unpinned, without--locked, and built on every PR. Now it's built only when the audit runs.securityis added toCI Gate'sneeds.Decisions (Nic, 2026-11-04):
Risk tier
Compatibility
Semver impact
Test evidence
actionlint .github/workflows/ci.yml: clean.I ran the script on dev2 (cargo-audit 0.22.1, advisory DB of 2026-11-03, 1403 advisories) with real lockfiles:
main= origin/main'sCargo.lock.pre260= the lockfile before fix(deps): bump tokio to 1.53.3 for RUSTSEC-2026-0386 #260 (tokio 1.53.1).webpki-vuln= main with rustls-webpki moved back to 0.103.15, which is before chore(deps): move rustls-webpki to 0.103.16 and webpki-root-certs to 1.0.10 for two RustSec advisories #258.bigint-moved= main with crypto-bigint 0.5.5 → 0.5.4.introduced by this change: RUSTSEC-2026-0382 rustls-webpki 0.103.15new informational advisory: unsound RUSTSEC-2026-0386 tokio 1.53.1warningcargo audit produced no reportOne bug turned up in testing and is fixed: with an empty base list, awk's
NR == FNRalso matched the PR's list, so the no-base path passed. The script now usesFILENAME == ARGV[1].Round 2 (5cae049), on Windows with cargo-audit 0.22.1. The script was run from a PR checkout that adds rustls-webpki 0.103.15 (RUSTSEC-2026-0382), with base = main's
Cargo.lock:.cargo/audit.tomlignores 0382 (Hermes's case)introduced by this change: RUSTSEC-2026-0382--configignores applied: RUSTSEC-2026-0387--configfile missingno audit configThe job's audit step, extracted from the YAML, run on simulated PR merge commits:
.cargo/audit.tomlwarning;ignores applied: RUSTSEC-2026-0387;actionlintis clean.This PR changes
ci.yml, so its own Security Audit runs the full audit path, with an unchanged lockfile. It should pass, with 0387 as a warning.New dependency
none. cargo-audit was already used; it's now pinned.
ADR
n/a
Mitigation / rollback
Revert the PR. The job goes back to a plain
cargo auditoutsideCI Gate.🤖 Generated with Claude Code