Skip to content

fix(conformance): classify evidence isolation failures - #305

Merged
BunsDev merged 4 commits into
mainfrom
fix/evidence-isolation-diagnostics
Sep 16, 2026
Merged

BunsDev merged 4 commits into
mainfrom
fix/evidence-isolation-diagnostics

Conversation

@BunsDev

@BunsDev BunsDev commented Sep 16, 2026

Copy link
Copy Markdown
Member

Protected run 35111662551 reached the Windows evidence isolation check and failed with a generic category, producing no Windows record. Linux and macOS records passed independent identity, timing, privacy, and all 197 assertion checks each.

This change splits the existing isolation rejection into 13 fixed categories covering opaque IDs, native credential state, and each operator root's snapshot, path, or digest. It preserves all acceptance predicates and exposes no paths, hashes, or input values. Both public wrappers allow only those exact categories. Workflow byte pins, authority bindings, documentation metadata, and the roadmap checkpoint follow the reviewed source.

Validation includes regression cases through the actual producer/public wrappers and rejection of arbitrary diagnostic suffixes, plus normal/heavy suites (2,115 passed, 98 skipped), typecheck, lint, and independent review. The signed reviewed head binds direct source 2dd79ec1505d9c12051552109244067b35813357; all 25 governed files and 10 production deltas were verified. This is instrumentation; the Windows cause and protected acceptance remain unproven. SDK rebinding, both scope rotations, and fresh protected validation follow the actual delivery merge.

Tracks #219; consolidation remains tracked in OpenCoven/sdk#45. Preserve Chat #86 and active worktrees. Land with a normal merge whose complete tree equals the reviewed head, as required by the validator.

The review finding about null snapshots is addressed: all six operator-root/before-after combinations now produce the fixed per-root invalid category. Red/green regression tests confirmed rejection still occurs through the actual public wrappers. Signed head 291698a369eb796397a567886dcd5d694fd62080 binds that correction; full exact-head CI remains required.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Handle null operator snapshots before dereferencing them so bounded per-root diagnostics are preserved.

Pull request overview

Adds privacy-preserving, fixed-category diagnostics for evidence-isolation failures while preserving validation predicates and updating conformance metadata.

Changes:

  • Adds 13 allowlisted isolation failure categories and regression coverage.
  • Updates conformance pins, workflow integrity metadata, and documentation.
  • Preserves public-wrapper diagnostic boundaries.
File summaries
File Reviewed changes
src/phase1-isolation-diagnostics.test.ts Tests diagnostic propagation and privacy.
src/phase1-evidence-runtime.test.ts Updates isolation failure expectations.
src/phase1-conformance-lock.test.ts Updates authority assertions.
scripts/phase1-schema-v2-producer.mjs Allows categorized producer diagnostics.
scripts/phase1-evidence-runtime.mjs Emits categorized isolation failures; null snapshots can still throw before the per-root invalid category is emitted.
scripts/phase1-conformance.mjs Allowlists public diagnostics.
phase1-conformance.lock.json Pins reviewed source and hashes.
docs/roadmap.md Records the diagnostics checkpoint.
docs/phase1-conformance.md Updates conformance metadata.
.github/workflows/client-v1-conformance.yml Updates integrity metadata.
Review details

Suppressed comments (1)

scripts/phase1-evidence-runtime.mjs:170

  • A malformed operator snapshot can contain null; the undefined checks do not cover that, so dereferencing before.sha256/after.sha256 throws a TypeError and the preflight falls back to generic ...isolation.failed instead of the new per-root .invalid category. Treat non-record snapshot entries as invalid before reading their digests so this diagnostic remains bounded for every invalid snapshot.
    if (
      before === undefined ||
      after === undefined ||
      !digestPattern.test(before.sha256 ?? '') ||
      !digestPattern.test(after.sha256 ?? '')
  • Files reviewed: 10/10 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:full Run the macOS and Windows CI jobs on this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants