Formalize parity tests as the darnit conformance suite (agent-agnostic contract)

[mlieberman85]

Summary

Feature 028 (Tier 1 parity, in-process) and feature 029 (Tier 2 parity, real Claude Agent SDK / OpenAI) established a working parity-test framework. Open issue #369 tracks scheduled cadence and governance-appropriate API key sourcing for Tier 2.

Time to elevate this from "parity" (implicitly Claude-shaped) to a documented darnit conformance suite: an agent-agnostic contract that any coding-agent-plus-skill implementation MUST honor to be a valid darnit consumer. Onboarding a new client (Codex, Cursor, OpenCode, per #388) then reduces to "run the conformance suite and ship the report."

Scope

• Contract: publish the per-control output schema (status, evidence shape, source enum, authority tag, etc.) as pydantic models under a stable module path. Both the MCP tool path and the harness path already produce this shape; the pydantic models codify it.
• Suite: absorb feature 028's Tier 1 tests and feature 029's Tier 2 tests into a single tests/darnit/conformance/ tree with a documented invocation contract (pytest -m conformance or similar).
• Cadence and key sourcing: absorb Parity Tier 2: add scheduled cadence + governance-appropriate API-key sourcing #369 into this issue's follow-up work.
• Reports: each run produces a conformance report (pydantic-typed) that a new-client-onboarding PR can attach as evidence.

Non-goals

• Certifying non-Anthropic/OpenAI providers on day one (add them one at a time via their own workflow, mirroring the Tier 2 pattern).
• Blocking every PR on the full conformance suite (Tier 2's Environment-gated model already handles the review-boundary problem).

Constitution alignment

• V. Sieve Pipeline Integrity: the conformance contract IS the sieve's user-facing invariant. Formalizing it makes the invariant testable at the framework boundary, not just at the handler boundary.
• III. TOML-First: the contract lives in code (pydantic) and docs, not in TOML -- but every field it references maps 1:1 to a TOML-defined control's declared output.

Related

• Test parity between coding-agent (MCP) audit path and harness audit path #366 (Test parity between coding-agent MCP audit path and harness audit path) -- CLOSED, feature 028's genesis.
• Parity: OpenAI (and other provider) skill-drift checks alongside the Claude Agent SDK version #368 (OpenAI parity) -- CLOSED, feature 029.
• Parity Tier 2: add scheduled cadence + governance-appropriate API-key sourcing #369 (Parity Tier 2: scheduled cadence + governance-appropriate API key sourcing) -- ABSORBED by this issue when it lands.
• Add Codex, Cursor, and OpenCode init options alongside Claude Code #388 (Codex/Cursor/OpenCode init) -- consumers of this contract.