fix: add falsification discipline to security audit template#243
Merged
Alan-Jowett merged 2 commits intomicrosoft:mainfrom Apr 14, 2026
Merged
Conversation
Three changes to reduce false positive findings in security audits: 1. Add adversarial-falsification protocol to investigate-security template. The template previously lacked the protocol that requires actively disproving each finding before reporting it. This omission allowed findings to be reported without checking for upstream validation in caller functions or API postconditions. 2. Add validation provenance check to security-vulnerability protocol (Phase 2, step 5). The protocol described forward tracing (entry point to use site) but not backward tracing (use site back to validation origin). The new step requires checking caller validation, API postconditions, and initialization-time invariants before reporting any finding. 3. Add symmetric falsification to self-verification protocol (Rule 1). The sampling verification step now requires attempting to disprove reported findings with the same rigor applied when falsifying candidates that were concluded safe — breaking confirmation bias toward reporting. Motivated by a real audit where 2 of 9 findings were false positives because upstream validation (in a caller function and a kernel API contract) was missed. The audit correctly falsified 6 other candidates as safe, demonstrating the capability existed but was applied asymmetrically. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds adversarial falsification rigor to the security audit workflow to reduce false-positive vulnerability findings by forcing backward validation tracing and symmetric verification standards.
Changes:
- Add
guardrails/adversarial-falsificationto theinvestigate-securitytemplate protocol list. - Extend the security vulnerability analysis protocol with a “validation provenance check” step (backward tracing to upstream validation/contracts).
- Strengthen
self-verificationsampling with a symmetric falsification requirement; sync the template’s protocol list inmanifest.yaml.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| templates/investigate-security.md | Adds guardrails/adversarial-falsification to the template’s protocol set. |
| protocols/guardrails/self-verification.md | Adds symmetric falsification guidance to sampling verification. |
| protocols/analysis/security-vulnerability.md | Adds backward-tracing validation provenance check before reporting findings. |
| manifest.yaml | Updates investigate-security protocol list to include adversarial-falsification. |
…equirements Address review feedback: 'confirmed no upstream validation' and 'answer no with code evidence' ask for proof of nonexistence, which is hard to satisfy consistently. Rephrase to require explicit documentation of what was checked, the evidence source, and why each check does not neutralize the finding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add adversarial falsification discipline to the security audit workflow to reduce false positive findings. Three targeted changes to the
investigate-securitytemplate and its protocols, motivated by a real-world audit where 2 of 9 findings were false positives due to missed upstream validation.Problem
During a security audit of a protocol implementation, the audit produced false positives because:
Incomplete provenance tracing: A finding reported an integer underflow at a use site, but an upstream caller function already validated the constraint. The audit analyzed the arithmetic in isolation without tracing back to the validation origin.
Missing API contract analysis: A finding reported a heap overflow via
memcpywith an unchecked length, but the length came fromZwQueryValueKeywhich guarantees the output fits within the provided buffer on success. The audit treated an API output as untrusted input.Both failures share a root cause: the audit applied falsification rigor asymmetrically — more rigorously to "this looks safe" conclusions (6 correctly falsified) than to "this looks dangerous" conclusions (2 false positives reported).
Changes
templates/investigate-security.mdguardrails/adversarial-falsificationto protocol listprotocols/analysis/security-vulnerability.mdprotocols/guardrails/self-verification.mdmanifest.yamlinvestigate-securityDesign Decisions
adversarial-falsificationprotocol — it already contains the exact rules needed (Rule 2: "Disprove Before Reporting", Rule 4: "Verify Helpers and Callers"). The template simply wasn't including it.security-vulnerabilityrather than creating a new protocol — the step is specific to input validation audit (Phase 2) and extends the existing forward-tracing methodology.self-verification— this is a cross-cutting guardrail change that benefits all templates, not just security audits. The language is minimal and scoped to the sampling verification step.Checklist