Repository navigation
ci: bind corpus headline deferral to the executing base - #870
luisleo526 wants to merge 2 commits into
Conversation
Preserve the trusted-base bridge and defer changed headlines only when both immutable base and head workflows contain the complete evidence contract. Cover OLD-base rejection and both-NEW acceptance with real Git fixtures.
| from pathlib import Path | ||
| import subprocess | ||
| import tempfile | ||
| import unittest |
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 2 advisory finding(s) below merit a look before merge.
Not checked: tests were not run; formal verification proved 0 of 3 changed function(s) (2 not verified).
Formal verification. 1 change(s) alter behavior, breaking input(s) attached. PR-changed functions: 1/3 verified (0 proven, 0 may-equivalent, 1 distinguished) · 2 not verified (2 vacuous).
Behavior changes: gke\_admit\_workflow\_findings changes behavior, here is the input that shows it.
The verifier found a concrete input on which gke\_admit\_workflow\_findings behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"workflow":"' '","name":"'gke\-pr\.yml'"\}, the old code produced \['gke\-pr\.yml: trigger must be exactly pull\_request\_target', 'gke\-pr\.yml: trigger must restrict branches to main', 'gke\-pr\.yml: trigger types are incomplete or reordered', 'gke\-pr\.yml: run\… but the new code produces \['gke\-pr\.yml result must match one complete OLD or NEW headline contract', 'gke\-pr\.yml: trigger must be exactly pull\_request\_target', 'gke\-pr\.yml: trigger must restrict branches to main', 'g… (outputs first differ at character 15). Paste that input straight into a regression test.
Not verified on this run: check\_commands (vacuous: never exercised), main (vacuous: never exercised).
Graphify review — findings
Adds check_corpus_headline_evidence.py, which runs from the trusted PR base and refuses a change to the EXPECTED_VERIFY headline default unless a recorded native run backs it. It hashes the tracked scientific inputs (pin normalized, docs and CI files excluded), then checks the record's source commit, probe population, zero verifier and native exits, native summary digest, headline bytes, and loader environment, treating candidate objects and PR attachments as data only. Extends ci_preflight.py and its tests so the checker's window, the GKE runner and arm-job policy, the promotion guard, and the prebuilt-image and result-classifier steps are enforced in preflight, with the classifier's Compare API lookup and the attachment handling in main being where the new checks could silently under-cover.
Worth a look
- Compare API results are silently limited to the first page —
scripts/ci_preflight.py:635· Escalate · high- unverified hypothesis — survived the adversarial refute pass; needs human review
- PR attachment is not revalidated after evidence checks —
scripts/check_corpus_headline_evidence.py:457· Escalate · medium- unverified hypothesis — survived the adversarial refute pass; needs human review
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 1153 functions depend on the 1056 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
gke_admit_workflow_findings()— 21 callers, 15 callees - worse:
main_admission_data_findings()— 10 callers, 3 callees - worse:
main()— 1 callers, 14 callees - new:
check_service()— 1 callers, 9 callees - new:
engine_identity()— 1 callers, 7 callees - new:
main()— 1 callers, 7 callees
Verification — 1153 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 1120 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
8 of 1436 test file(s) selected (1%) via static blast radius.
scripts/test_check_corpus_headline_evidence.py— impact, changed-testscripts/test_check_doc_reverts.py— impactscripts/test_ci_corpus_variant.py— impact, test-dependencyscripts/test_ci_preflight.py— impact, changed-testscripts/test_ci_preflight_split.py— test-dependencyscripts/test_ci_release_gate.py— impactscripts/test_cr150_source_gate.py— impacttests/test_report_sink_format.cpp— impact
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Risky patterns (advisory)
- medium
no-stderr-to-stdoutinscripts/check_corpus_headline_evidence.py:464: error printed to stdout instead of stderr: print("corpus headline evidence refused: " + " ".join(str(error).splitlines()))
Docs that may be stale (advisory)
AGENTS.md§ What this repo is (lines 5-11): references changed symbolssourceAGENTS.md§ Build (lines 12-25): references changed symbolsgitAGENTS.md§ Layout (lines 26-41): references changed symbolssourceAGENTS.md§ What CI runs, per trigger (lines 46-74): references changed symbolscommitAGENTS.md§ Conventions (lines 107-120): references changed symbolssourceAGENTS.md§ Safety rules (lines 121-131): references changed symbolssourceCHANGELOG.md§ Unreleased (lines 8-255): references changed symbolsrequest,run,snapshotCHANGELOG.md§ 1.9.1 — 2026-10-31 (lines 256-378): references changed symbolscommit,constant,sha256,sourceCHANGELOG.md§ Closed-trade reads (lines 379-429): references changed symbolsfixtureCHANGELOG.md§ Live ledger report-checkpoint layouts (lines 466-473): references changed symbolsrun
…and 10 more.
Formal verification
Behavior changes: gke\_admit\_workflow\_findings changes behavior, here is the input that shows it.
The verifier found a concrete input on which gke\_admit\_workflow\_findings behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"workflow":"' '","name":"'gke\-pr\.yml'"\}, the old code produced \['gke\-pr\.yml: trigger must be exactly pull\_request\_target', 'gke\-pr\.yml: trigger must restrict branches to main', 'gke\-pr\.yml: trigger types are incomplete or reordered', 'gke\-pr\.yml: run\… but the new code produces \['gke\-pr\.yml result must match one complete OLD or NEW headline contract', 'gke\-pr\.yml: trigger must be exactly pull\_request\_target', 'gke\-pr\.yml: trigger must restrict branches to main', 'g… (outputs first differ at character 15). Paste that input straight into a regression test.
Could not verify: Could not verify check\_commands.
The verifier did not have enough to check check\_commands, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 90 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly NameError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify main.
The verifier did not have enough to check main, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 1 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly NameError — names the real obstacle, not a sampling gap)
· 3 grounded finding(s) anchored inline below; 3 more finding(s) on lines outside this diff (see the check run).
| or ("/" not in path and path.endswith(".md"))) | ||
|
|
||
|
|
||
| def engine_identity(repo, revision): |
There was a problem hiding this comment.
engine_identity()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return (json_object(archive.read(names[0])), archive.read(names[1]), archive.read(names[2])) | ||
|
|
||
|
|
||
| def check_service(service, attached, repo, base, head, expected): |
There was a problem hiding this comment.
check_service()
fans out to 9 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return total | ||
|
|
||
|
|
||
| def main(argv=None): |
There was a problem hiding this comment.
main()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
Use the validated GITHUB_SHA and matching checkout HEAD as the executing base for pull_request_target admission. Keep the payload base SHA as informational data so a stale PR payload cannot reject a valid event. Preserve canonical fetch, full-history, immutable-object and workflow contract checks, with real Git regressions for stale and equal bases.
Bind the corpus headline deferral to the executing base.
The trusted-base bridge is kept. A changed headline is deferred only when both the immutable base workflow and the head workflow contain the complete evidence contract. The admission check requires the hosted
GITHUB_SHAto equal the pull request's base commit and reads the canonical repository's full history.Tests cover rejection on an old base and acceptance when both sides carry the new contract, using real Git fixtures.
This pull request is also the first hosted run of the admission check itself: its own CI shows whether the hosted commit and the base commit agree and whether the full-history fetch succeeds, including after a ready-for-review transition.
🤖 Generated with Claude Code