Repository navigation
Bind release reports to exact attempts and prove empty cancellations - #182
Conversation
Refuse absent, invalid or mismatched release snapshots before transport and retain full receipt pin checks. Keep failure incidents separate from release publication and correct publisher-attempt fixtures. Native proof is pending.
Carry private producer outcomes through release stages and independently validate the current run, artifact inventory, jobs and decision inputs. Keep stored verified receipt identities intact and avoid re-emitting replay. Exercise the production CLI paths with independent transport observations, retain missing-target refusals, and keep public receipt/callback contracts. Native fail-first checks and governing Claude acceptance remain required.
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.
Not checked: tests were not run; formal verification proved 0 of 9 changed function(s) (1 sampled, not proven, 8 not verified).
Formal verification. PR-changed functions: 1/9 verified (0 proven, 1 may-equivalent, 0 distinguished) · 8 not verified (8 vacuous).
Not verified on this run: finish (vacuous: never exercised), reconcile (vacuous: never exercised), main (vacuous: never exercised), prepare (vacuous: never exercised), recover\_applied (vacuous: never exercised), collect (vacuous: never exercised), main (vacuous: never exercised), publication (vacuous: never exercised).
Graphify review — findings
Adds a private producer-outcome.json witness to each release stage artifact and a run-context.json capture in the reporting workflow, so the reporter distinguishes a targetless no-op, a stored verified replay and current targeted output instead of inferring success from an empty inventory. The reporter binds both files to the API-observed run, attempt, head and job inventory, and missing or conflicting evidence refuses collection rather than granting a quiet pass. The release report job now also runs for placeholder claims by adding claim-name to needs, and the if: !cancelled() guard on it means a cancelled run can skip reporting entirely, while cancelled jobs fall through to the generic publication stage label. The incident path now calls a dedicated incident subcommand and carries the caller's real failed and stage values rather than hardcoding receipt-transport. The unconditional has_target guards on the stage-artifact downloads are removed so the witness is always fetched, and docs/release-receipt-report.md is rewritten around the new contract.
Worth a look
- Cancellation suppresses the report job —
.github/workflows/release.yml:473· Escalate · medium- unverified hypothesis — survived the adversarial refute pass; needs human review
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 551 functions depend on the 549 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
publication()— 14 callers, 30 callees - worse:
verified_package()— 25 callers, 15 callees - worse:
recover_applied()— 17 callers, 17 callees - worse:
reconcile()— 8 callers, 27 callees - worse:
main()— 13 callers, 13 callees - worse:
prepare()— 3 callers, 19 callees - worse:
main()— 4 callers, 14 callees - new:
no_producer_facts()— 13 callers, 3 callees - …and 28 more — each is listed as a finding
Verification — 551 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 551 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
23 of 23 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety (
noncode-change,uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_backtest.py— full-run-safetytests/test_ccxt.py— full-run-safetytests/test_compile_cache.py— full-run-safetytests/test_consumer_receipt.py— impact, changed-test, full-run-safetytests/test_database_e2e.py— full-run-safetytests/test_docker_integration.py— full-run-safetytests/test_docker_runtime.py— full-run-safetytests/test_documentation.py— full-run-safetytests/test_engine.py— full-run-safetytests/test_fxmacrodata.py— full-run-safetytests/test_main_proofs.py— full-run-safetytests/test_models.py— full-run-safetytests/test_post_merge_workflows.py— full-run-safetytests/test_pr_tier_workflows.py— full-run-safetytests/test_providers.py— full-run-safetytests/test_release.py— impact, full-run-safetytests/test_release_adapter.py— impact, changed-test, full-run-safetytests/test_release_contract.py— full-run-safetytests/test_release_reporting.py— impact, changed-test, full-run-safetytests/test_release_workflow_authz.py— impact, changed-test, full-run-safetytests/test_server.py— full-run-safetytests/test_server_client.py— full-run-safetytests/test_tabular_providers.py— full-run-safety
non-code file(s) changed (
.github/workflows/pineforge-release.yml,.github/workflows/release-report.yml,.github/workflows/release.yml,docs/release-receipt-report.md,docs/releasing.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
.github/workflows/pineforge-release.yml,.github/workflows/release-report.yml,docs/release-receipt-report.md,docs/releasing.md) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Docs that may be stale (advisory)
README.md§ Contributing (lines 145-169): references changed symbolsgitdocs/backtesting.md§ CLI reference (lines 26-53): references changed symbolsapidocs/provider-contract.md§ In-tree adapter skeleton (lines 32-81): references changed symbols__init__,requestdocs/providers.md§ Common market and bar API (lines 74-111): references changed symbolsfetch,requestdocs/providers/fxmacrodata.md§ Timestamps and vintages (lines 52-86): references changed symbolssnapshotdocs/providers/fxmacrodata.md§ Constructor reference (lines 107-128): references changed symbolsapinative/feed/README.md§ Prints-built history: export (lines 91-165): references changed symbolsarchivenative/feed/README.md§ Serve: one producer, many runners (lines 541-617): references changed symbolssnapshottests/fixtures/release_chain/README.md§ Offline release-chain fixtures (lines 1-21): references changed symbolsreceipt
Formal verification
Could not verify: Could not verify finish.
The verifier did not have enough to check finish, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly ValueError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify reconcile.
The verifier did not have enough to check reconcile, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 189 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly CalledProcessError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify main.
The verifier did not have enough to check main, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 1 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify prepare.
The verifier did not have enough to check prepare, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 253 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly FileNotFoundError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify recover\_applied.
The verifier did not have enough to check recover\_applied, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 225 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly FileNotFoundError — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in receipt (not a proof).
The verifier ran both versions of receipt on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify collect.
The verifier did not have enough to check collect, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 9 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify main.
The verifier did not have enough to check main, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 1 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify publication.
The verifier did not have enough to check publication, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 253 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly ValueError — names the real obstacle, not a sampling gap)
· 9 grounded finding(s) anchored inline below; 27 more finding(s) on lines outside this diff (see the check run).
| @@ -91,6 +100,10 @@ def body(metadata: dict[str, Any]) -> str: | |||
|
|
|||
| def finish(report: dict[str, Any], path: Path) -> None: | |||
There was a problem hiding this comment.
finish()
11 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
|
||
|
|
||
| def recover_applied(repo: str, source: Path, report: dict[str, Any]) -> None: | ||
| def recover_applied( |
There was a problem hiding this comment.
recover_applied()
fans out to 17 callees (efferent coupling); 17 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| raise ValueError("release-snapshot-invalid: unreadable stage archive") from error | ||
|
|
||
|
|
||
| def checked_no_target( |
There was a problem hiding this comment.
checked_no_target()
fans out to 15 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| checked_handoff_target(repo, publisher, inputs["version"]) | ||
|
|
||
|
|
||
| def checked_handoff_target( |
There was a problem hiding this comment.
checked_handoff_target()
fans out to 9 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| ) | ||
|
|
||
|
|
||
| def checked_stage( |
There was a problem hiding this comment.
checked_stage()
fans out to 12 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
|
||
|
|
||
| def publication(repo: str, run_id: int, report: dict[str, Any]) -> None: | ||
| def publication( |
There was a problem hiding this comment.
publication()
fans out to 30 callees (efferent coupling); 14 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -330,29 +1183,63 @@ def retain(repo: str, report: dict[str, Any]) -> None: | |||
|
|
|||
| def main() -> None: | |||
There was a problem hiding this comment.
main()
fans out to 13 callees (efferent coupling); 13 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| None, "missing", "run", "attempt", "head", "schema", "fields", | ||
| "ci-run", "ci-attempt", "source", "version", | ||
| ]) | ||
| def test_cli_targetless_package_carries_actual_outcome_through_every_publisher_stage( |
There was a problem hiding this comment.
test_cli_targetless_package_carries_actual_outcome_through_every_publisher_stage()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| assert all(method == "GET" for _, method in state["cli_calls"]) | ||
|
|
||
|
|
||
| def test_cli_recovery_handoff_uses_real_gates_and_emits_no_new_receipt( |
There was a problem hiding this comment.
test_cli_recovery_handoff_uses_real_gates_and_emits_no_new_receipt()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
Sort imports, wrap long lines, mark an unused loop key, make existing boolean grouping explicit, retain regex values as raw literals, and preserve list order with unpacking. Keep every fixture, parameter case, assertion and release guard; no rule/configuration exemptions. ONE repair member on 590d8e6 for PR182's Ruff failure in run43744978020. New exact-head Ruff and the four-module native suite must both pass before CORE receives the publication-qualified member bundle.
Report-only release reruns can lose their original trigger/attempt or accept incomplete cancellation evidence. This change fetches the original event and exact attempt before publisher inspection, rejects missing or mismatched evidence, and allows a terminal cancelled Release/main run only when complete enumeration proves no executions and no artifacts. Latest-run movement stays a separate observation.
Validation on the frozen candidate: 1819 tests passed, 0 failed or errored, 1 existing opt-in live-PyPI skip; the 14 previous fixture failures and the four original causal cases pass. One source review completed with no blocking findings. Live provider behaviour remains unobserved here and hosted CI on this head is pending; this PR closes no public issue.
🤖 Generated with Claude Code