From c1313c668e407886dd812fe2f828be39c03354c2 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:15:54 +0000 Subject: [PATCH 1/3] =?UTF-8?q?invoker:=20wf-1789236097612-18/implement-sw?= =?UTF-8?q?eep-landing-tally=20=E2=80=94=20Make=20Step=206=20compute=20its?= =?UTF-8?q?=20merged=20tally=20from=20the=20trunk=20ancestry=20check.=20Re?= =?UTF-8?q?view=20claim:=20A=20sweep=20cannot=20report=20a=20PR=20as=20mer?= =?UTF-8?q?ged=20unless=20its=20merge=20commit=20is=20an=20ancestor=20of?= =?UTF-8?q?=20the=20trunk.=20Review=20lane:=20behavior=20Safety=20invarian?= =?UTF-8?q?t:=20The=20merged=20tally=20cannot=20be=20computed=20from=20`gh?= =?UTF-8?q?=20pr=20view`=20state=20alone;=20a=20PR=20whose=20ancestry=20ch?= =?UTF-8?q?eck=20does=20not=20return=20OK=20is=20reported=20unchecked,=20n?= =?UTF-8?q?ot=20merged.=20Effectiveness=20measurement:=20A=20fixture=20PR?= =?UTF-8?q?=20that=20closed=20against=20a=20stack=20branch=20rather=20than?= =?UTF-8?q?=20the=20trunk=20must=20be=20reported=20unchecked=20by=20the=20?= =?UTF-8?q?new=20Step=206=20and=20merged=20by=20the=20old=20one,=20asserte?= =?UTF-8?q?d=20in=20the=20skill's=20test=20directory.=20Slice=20rationale:?= =?UTF-8?q?=20One=20step=20of=20the=20sweep=20skill's=20reporting=20proced?= =?UTF-8?q?ure;=20stacked=20behind=20the=20probe=20step=20because=20both?= =?UTF-8?q?=20edit=20the=20same=20SKILL.md.=20Architectural=20effect:=20St?= =?UTF-8?q?ep=206=20gains=20a=20dependency=20on=20the=20already-shipped=20?= =?UTF-8?q?verify=5Fpr=5Flanded=5Fon=5Ftrunk.sh=20instead=20of=20relying?= =?UTF-8?q?=20on=20gh=20pr=20view=20alone.=20Goal:=20Rewrite=20Step=206=20?= =?UTF-8?q?of=20product/skills/admin-bypass-sweep/SKILL.md=20so=20each=20P?= =?UTF-8?q?R's=20status=20line=20comes=20from=20engine/hooks/gh-write-veri?= =?UTF-8?q?fication/verify=5Fpr=5Flanded=5Fon=5Ftrunk.sh=20and=20a=20non-O?= =?UTF-8?q?K=20result=20is=20reported=20as=20unchecked.=20Motivation:=20A?= =?UTF-8?q?=2019-PR=20sweep=20reported=2013=20merged=20from=20gh=20pr=20vi?= =?UTF-8?q?ew=20output=20alone.=20A=20Stop=20hook=20had=20to=20force=20the?= =?UTF-8?q?=20real=20ancestry=20check=20afterwards.=20All=2013=20did=20pas?= =?UTF-8?q?s,=20but=20the=20claim=20went=20out=20before=20the=20check=20th?= =?UTF-8?q?at=20proves=20it,=20and=20the=20same=20skill=20had=20already=20?= =?UTF-8?q?caused=20a=20manual=20replay=20of=20four=20PRs=20three=20days?= =?UTF-8?q?=20earlier.=20Alternative=20considerations:=20Adding=20a=20sent?= =?UTF-8?q?ence=20telling=20the=20operator=20to=20also=20run=20the=20verif?= =?UTF-8?q?ier=20was=20rejected:=20Step=206=20already=20says=20to=20re-que?= =?UTF-8?q?ry=20every=20PR=20and=20not=20trust=20running=20tallies,=20and?= =?UTF-8?q?=20that=20prose=20did=20not=20produce=20an=20ancestry=20check.?= =?UTF-8?q?=20Leaving=20the=20hook=20as=20the=20only=20catch=20was=20rejec?= =?UTF-8?q?ted=20because=20it=20fires=20after=20the=20reply=20is=20drafted?= =?UTF-8?q?.=20Implementation=20details:=20Replace=20the=20`gh=20pr=20view?= =?UTF-8?q?=20--json=20number,state,mergedAt,title`=20loop=20in=20Step=206?= =?UTF-8?q?=20with=20a=20loop=20calling=20verify=5Fpr=5Flanded=5Fon=5Ftrun?= =?UTF-8?q?k.sh=20per=20PR,=20branching=20on=20its=20documented=20exit=20c?= =?UTF-8?q?odes:=20engine/hooks/gh-write-verification/verify=5Fpr=5Flanded?= =?UTF-8?q?=5Fon=5Ftrunk.sh=20lines=204-6=20(EXIT=5FOK=3D0,=20EXIT=5FFAIL?= =?UTF-8?q?=3D1,=20EXIT=5FUNCHECKED=3D3).=20Report=20exit=200=20as=20merge?= =?UTF-8?q?d,=20exit=201=20as=20merged-but-not-on-trunk,=20and=20exit=203?= =?UTF-8?q?=20as=20unchecked.=20State=20in=20the=20prose=20that=20a=20merg?= =?UTF-8?q?ed=20count=20may=20not=20be=20reported=20from=20gh=20pr=20view?= =?UTF-8?q?=20alone.=20Add=20a=20fixture=20under=20the=20skill's=20tests?= =?UTF-8?q?=20directory=20asserting=20the=20new=20Step=206=20text=20names?= =?UTF-8?q?=20the=20verifier=20and=20the=20three=20outcomes.=20Non-goals:?= =?UTF-8?q?=20Does=20not=20change=20the=20merge=20commands=20in=20Step=204?= =?UTF-8?q?,=20does=20not=20change=20Step=205's=20blocked-PR=20policy,=20d?= =?UTF-8?q?oes=20not=20modify=20the=20verifier=20script=20itself,=20and=20?= =?UTF-8?q?does=20not=20touch=20any=20other=20skill.=20Layer:=20domain=20F?= =?UTF-8?q?eature=20state:=20active=20Files:=20product/skills/admin-bypass?= =?UTF-8?q?-sweep/SKILL.md,=20product/skills/admin-bypass-sweep/tests/=20C?= =?UTF-8?q?hange=20types:=20-=20product/skills/admin-bypass-sweep/SKILL.md?= =?UTF-8?q?:=20docs-only=20-=20product/skills/admin-bypass-sweep/tests/fir?= =?UTF-8?q?es=5Fstep6=5Fnames=5Fverifier.md:=20create=20Acceptance=20crite?= =?UTF-8?q?ria:=20-=20`python3=20-m=20unittest=20discover=20-s=20product/s?= =?UTF-8?q?kills/admin-bypass-sweep/tests=20-v`=20exits=200.=20-=20Step=20?= =?UTF-8?q?6=20names=20verify=5Fpr=5Flanded=5Fon=5Ftrunk.sh=20and=20all=20?= =?UTF-8?q?three=20exit=20codes.=20-=20Step=206=20no=20longer=20derives=20?= =?UTF-8?q?a=20merged=20count=20from=20`gh=20pr=20view`=20state=20alone.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Solution: Make Step 6 compute its merged tally from the trunk ancestry check. Review claim: A sweep cannot report a PR as merged unless its merge commit is an ancestor of the trunk. Review lane: behavior Safety invariant: The merged tally cannot be computed from `gh pr view` state alone; a PR whose ancestry check does not return OK is reported unchecked, not merged. Effectiveness measurement: A fixture PR that closed against a stack branch rather than the trunk must be reported unchecked by the new Step 6 and merged by the old one, asserted in the skill's test directory. Slice rationale: One step of the sweep skill's reporting procedure; stacked behind the probe step because both edit the same SKILL.md. Architectural effect: Step 6 gains a dependency on the already-shipped verify_pr_landed_on_trunk.sh instead of relying on gh pr view alone. Goal: Rewrite Step 6 of product/skills/admin-bypass-sweep/SKILL.md so each PR's status line comes from engine/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh and a non-OK result is reported as unchecked. Motivation: A 19-PR sweep reported 13 merged from gh pr view output alone. A Stop hook had to force the real ancestry check afterwards. All 13 did pass, but the claim went out before the check that proves it, and the same skill had already caused a manual replay of four PRs three days earlier. Alternative considerations: Adding a sentence telling the operator to also run the verifier was rejected: Step 6 already says to re-query every PR and not trust running tallies, and that prose did not produce an ancestry check. Leaving the hook as the only catch was rejected because it fires after the reply is drafted. Implementation details: Replace the `gh pr view --json number,state,mergedAt,title` loop in Step 6 with a loop calling verify_pr_landed_on_trunk.sh per PR, branching on its documented exit codes: engine/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh lines 4-6 (EXIT_OK=0, EXIT_FAIL=1, EXIT_UNCHECKED=3). Report exit 0 as merged, exit 1 as merged-but-not-on-trunk, and exit 3 as unchecked. State in the prose that a merged count may not be reported from gh pr view alone. Add a fixture under the skill's tests directory asserting the new Step 6 text names the verifier and the three outcomes. Non-goals: Does not change the merge commands in Step 4, does not change Step 5's blocked-PR policy, does not modify the verifier script itself, and does not touch any other skill. Layer: domain Feature state: active Files: product/skills/admin-bypass-sweep/SKILL.md, product/skills/admin-bypass-sweep/tests/ Change types: - product/skills/admin-bypass-sweep/SKILL.md: docs-only - product/skills/admin-bypass-sweep/tests/fires_step6_names_verifier.md: create Acceptance criteria: - `python3 -m unittest discover -s product/skills/admin-bypass-sweep/tests -v` exits 0. - Step 6 names verify_pr_landed_on_trunk.sh and all three exit codes. - Step 6 no longer derives a merged count from `gh pr view` state alone. Invoker-Finalize-Id: 616fd5ec-4b0c-4737-9ac1-85da4a7c9f4a --- product/skills/admin-bypass-sweep/SKILL.md | 42 ++++++++++++--- .../tests/stack_branch_landing_fixture.md | 29 ++++++++++ .../tests/test_step_6_verifier_tally.py | 53 +++++++++++++++++++ 3 files changed, 118 insertions(+), 6 deletions(-) create mode 100644 product/skills/admin-bypass-sweep/tests/stack_branch_landing_fixture.md create mode 100644 product/skills/admin-bypass-sweep/tests/test_step_6_verifier_tally.py diff --git a/product/skills/admin-bypass-sweep/SKILL.md b/product/skills/admin-bypass-sweep/SKILL.md index becb9df3..bc6b6ec1 100644 --- a/product/skills/admin-bypass-sweep/SKILL.md +++ b/product/skills/admin-bypass-sweep/SKILL.md @@ -223,18 +223,48 @@ fate — merged or genuinely blocked — is decided. ## Step 6: Prove the final state Before reporting results, re-query every PR number touched — do not trust -running tallies kept during execution: +running tallies kept during execution. A merged tally may not be computed +from `gh pr view` state alone; GitHub's PR state is not proof that the PR's +merge commit reached the trunk. ```bash +verify="engine/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh" +trunk="" # for example, master + for pr in ; do - gh pr view $pr --repo / --json number,state,mergedAt,title \ - --jq '"#\(.number)\t\(.state)\t\(.mergedAt // "-")\t\(.title)"' + title="$(gh pr view "$pr" --repo / --json number,title \ + --jq '"#\(.number)\t\(.title)"')" || title="#${pr}\t" + + verifier_status=0 + verifier_output="$("$verify" --repo <owner>/<repo> "$pr" "$trunk" 2>&1)" || verifier_status=$? + + case "$verifier_status" in + 0) + printf '%s\tmerged\t%s\n' "$title" "$verifier_output" + ;; + 1) + printf '%s\tmerged-but-not-on-trunk\t%s\n' "$title" "$verifier_output" + ;; + 3) + printf '%s\tunchecked\t%s\n' "$title" "$verifier_output" + ;; + *) + printf '%s\tunchecked\tverify_pr_landed_on_trunk.sh exited %s: %s\n' \ + "$title" "$verifier_status" "$verifier_output" + ;; + esac done ``` -Report merged count, blocked count, and the specific PR numbers/titles in -each bucket — a summary count alone hides which specific work is still -stuck. +Branch only on `engine/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh` +exit codes: exit 0 is `merged`, exit 1 is `merged-but-not-on-trunk`, and +exit 3 is `unchecked`. Exit 3 is never reported as merged. Only exit 0 +increments the merged count; exit 1 and exit 3 must be counted in their own +buckets and named with their PR numbers/titles. + +Report merged count, merged-but-not-on-trunk count, unchecked count, blocked +count, and the specific PR numbers/titles in each bucket — a summary count +alone hides which specific work is still stuck. ## Why this exists diff --git a/product/skills/admin-bypass-sweep/tests/stack_branch_landing_fixture.md b/product/skills/admin-bypass-sweep/tests/stack_branch_landing_fixture.md new file mode 100644 index 00000000..78afb031 --- /dev/null +++ b/product/skills/admin-bypass-sweep/tests/stack_branch_landing_fixture.md @@ -0,0 +1,29 @@ +# Stack-branch landing fixture + +PR #1789 was reported by GitHub as merged after it landed against a stack +branch instead of the trunk. + +Old Step 6 input: + +```text +#1789 MERGED 2026-09-09T10:11:12Z Retarget child stack branch +``` + +Old Step 6 outcome: + +```text +merged +``` + +New Step 6 verifier result: + +```text +exit 1 +FAIL: PR #1789 in owner/repo reports MERGED but abc123 is not on origin/master +``` + +New Step 6 outcome: + +```text +merged-but-not-on-trunk +``` diff --git a/product/skills/admin-bypass-sweep/tests/test_step_6_verifier_tally.py b/product/skills/admin-bypass-sweep/tests/test_step_6_verifier_tally.py new file mode 100644 index 00000000..486b8409 --- /dev/null +++ b/product/skills/admin-bypass-sweep/tests/test_step_6_verifier_tally.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import re +import unittest +from pathlib import Path + +SKILL_DIR = Path(__file__).resolve().parents[1] +SKILL = (SKILL_DIR / "SKILL.md").read_text(encoding="utf-8") +FIXTURE = (Path(__file__).resolve().parent / "stack_branch_landing_fixture.md").read_text(encoding="utf-8") + + +def step_6() -> str: + match = re.search(r"^## Step 6:.*?(?=^## )", SKILL, re.M | re.S) + if match is None: + raise AssertionError("Step 6 section not found") + return match.group(0) + + +class TestStep6VerifierTally(unittest.TestCase): + def setUp(self) -> None: + self.step = step_6() + self.compact_step = " ".join(self.step.split()) + + def test_step_6_names_the_trunk_ancestry_verifier(self): + self.assertIn("engine/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh", self.step) + + def test_step_6_maps_all_documented_status_codes(self): + for code, outcome in ( + ("exit 0", "merged"), + ("exit 1", "merged-but-not-on-trunk"), + ("exit 3", "unchecked"), + ): + self.assertIn(code, self.step) + self.assertIn(outcome, self.step) + + def test_gh_pr_view_state_alone_cannot_drive_the_merged_tally(self): + self.assertIn("A merged tally may not be computed from `gh pr view` state alone", self.compact_step) + self.assertNotIn("state,mergedAt", self.step) + self.assertIn("Only exit 0 increments the merged count", self.compact_step) + + def test_exit_3_is_never_merged(self): + self.assertIn("Exit 3 is never reported as merged", self.step) + + def test_stack_branch_fixture_is_no_longer_counted_as_merged(self): + self.assertIn("#1789\tMERGED", FIXTURE) + self.assertIn("Old Step 6 outcome:\n\n```text\nmerged\n```", FIXTURE) + self.assertIn("exit 1", FIXTURE) + self.assertIn("New Step 6 outcome:\n\n```text\nmerged-but-not-on-trunk\n```", FIXTURE) + + +if __name__ == "__main__": + unittest.main() From 52b2f59736e205c76613d97df8a2b3933b5bded8 Mon Sep 17 00:00:00 2001 From: Invoker Bot <invoker@local> Date: Sat, 12 Sep 2026 18:16:39 +0000 Subject: [PATCH 2/3] =?UTF-8?q?invoker:=20wf-1789236097612-18/verify-sweep?= =?UTF-8?q?-landing-tally=20=E2=80=94=20Run=20the=20deterministic=20proof?= =?UTF-8?q?=20for=20the=20trunk-derived=20merged=20tally.=20Review=20claim?= =?UTF-8?q?:=20The=20sweep=20skill's=20fixtures=20assert=20Step=206=20repo?= =?UTF-8?q?rts=20from=20the=20ancestry=20check.=20Review=20lane:=20proof?= =?UTF-8?q?=20Safety=20invariant:=20Proof-only;=20adds=20no=20product=20be?= =?UTF-8?q?havior.=20Effectiveness=20measurement:=20The=20suite=20fails=20?= =?UTF-8?q?if=20Step=206=20stops=20naming=20the=20verifier=20or=20drops=20?= =?UTF-8?q?the=20unchecked=20outcome.=20Slice=20rationale:=20One=20proof?= =?UTF-8?q?=20slice=20for=20this=20step.=20Architectural=20effect:=20None;?= =?UTF-8?q?=20verification=20only.=20Goal:=20Prove=20the=20trunk-derived?= =?UTF-8?q?=20merged=20tally=20deterministically.=20Motivation:=20The=20de?= =?UTF-8?q?fect=20is=20a=20reporting=20path,=20so=20the=20proof=20asserts?= =?UTF-8?q?=20the=20reporting=20text=20names=20its=20source.=20Alternative?= =?UTF-8?q?=20considerations:=20Manual=20verification=20was=20rejected=20a?= =?UTF-8?q?s=20non-deterministic.=20Implementation=20details:=20Execute=20?= =?UTF-8?q?the=20command=20below=20as=20the=20terminal=20proof.=20Non-goal?= =?UTF-8?q?s:=20No=20product=20edits=20here;=20proof=20only.=20Layer:=20ap?= =?UTF-8?q?p=5Fregression=20Feature=20state:=20active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 931b523d-2572-41c1-8c3e-4f3e187d758f From 253d0deaf5f69a16a30db1029beedee9dd018241 Mon Sep 17 00:00:00 2001 From: Invoker Bot <invoker@local> Date: Sat, 12 Sep 2026 18:17:25 +0000 Subject: [PATCH 3/3] =?UTF-8?q?invoker:=20wf-1789236097612-18/scrub-handof?= =?UTF-8?q?f-artifacts=20=E2=80=94=20Terminal=20check=20that=20no=20epheme?= =?UTF-8?q?ral=20inter-task=20handoff=20files=20remain.=20Review=20claim:?= =?UTF-8?q?=20The=20workflow=20leaves=20no=20scratch=20handoff=20artifacts?= =?UTF-8?q?=20behind.=20Review=20lane:=20cleanup=20Safety=20invariant:=20R?= =?UTF-8?q?ead-only;=20never=20deletes=20files,=20alters=20the=20index,=20?= =?UTF-8?q?or=20commits=20caller=20work.=20Effectiveness=20measurement:=20?= =?UTF-8?q?The=20gate=20fails=20when=20a=20plans/invoker-handoff.*=20or=20?= =?UTF-8?q?lens-*.json=20file=20is=20still=20present=20after=20the=20leaf?= =?UTF-8?q?=20tasks=20complete.=20Slice=20rationale:=20One=20terminal=20hy?= =?UTF-8?q?giene=20gate=20for=20the=20workflow.=20Architectural=20effect:?= =?UTF-8?q?=20None;=20check=20only.=20Goal:=20Confirm=20no=20ephemeral=20h?= =?UTF-8?q?andoff=20files=20survive=20the=20run.=20Motivation:=20Inter-tas?= =?UTF-8?q?k=20scratch=20files=20leak=20into=20diffs=20and=20read=20as=20p?= =?UTF-8?q?art=20of=20the=20work.=20Alternative=20considerations:=20Deleti?= =?UTF-8?q?ng=20them=20automatically=20was=20rejected;=20the=20gate=20repo?= =?UTF-8?q?rts,=20it=20does=20not=20mutate.=20Implementation=20details:=20?= =?UTF-8?q?Run=20scripts/scrub-handoff-artifacts.sh=20without=20--apply.?= =?UTF-8?q?=20Non-goals:=20No=20deletion,=20no=20index=20changes,=20no=20c?= =?UTF-8?q?ommits.=20Layer:=20app=5Fregression=20Feature=20state:=20active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 9d4fd2eb-452d-4846-91e9-4fe370c27a64