From f04c9d76b20ad8a87eb7e0a4eb2da644c44ee5d4 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:08:40 +0000 Subject: [PATCH 1/3] =?UTF-8?q?invoker:=20wf-1789236146378-22/implement-la?= =?UTF-8?q?nding-detector-result=20=E2=80=94=20Make=20the=20landing-proof?= =?UTF-8?q?=20detector=20require=20the=20check's=20OK=20result,=20not=20me?= =?UTF-8?q?rely=20its=20invocation.=20Review=20claim:=20A=20landing=20chec?= =?UTF-8?q?k=20that=20printed=20FAIL=20or=20UNCHECKED=20does=20not=20satis?= =?UTF-8?q?fy=20the=20landing-proof=20hook.=20Review=20lane:=20behavior=20?= =?UTF-8?q?Safety=20invariant:=20No=20merge=20shape=20that=20is=20silent?= =?UTF-8?q?=20today=20becomes=20noisy:=20a=20merge=20followed=20by=20a=20l?= =?UTF-8?q?anding=20check=20printing=20OK=20stays=20silent,=20and=20only?= =?UTF-8?q?=20FAIL,=20UNCHECKED,=20or=20a=20missing=20result=20newly=20fla?= =?UTF-8?q?gs.=20Effectiveness=20measurement:=20Three=20fixtures:=20a=20la?= =?UTF-8?q?nding=20check=20printing=20OK=20stays=20silent,=20one=20printin?= =?UTF-8?q?g=20FAIL=20flags,=20one=20printing=20UNCHECKED=20flags.=20The?= =?UTF-8?q?=20current=20detector=20passes=20only=20the=20first,=20so=20the?= =?UTF-8?q?=20suite=20fails=20before=20the=20change=20and=20passes=20after?= =?UTF-8?q?.=20Slice=20rationale:=20One=20detector=20and=20its=20tests=20i?= =?UTF-8?q?n=20one=20hook=20directory;=20shares=20no=20file=20with=20the?= =?UTF-8?q?=20sweep-skill=20chain,=20so=20it=20runs=20independently=20rath?= =?UTF-8?q?er=20than=20waiting=20on=20it.=20Architectural=20effect:=20The?= =?UTF-8?q?=20detector=20gains=20access=20to=20each=20command's=20paired?= =?UTF-8?q?=20tool=20result,=20so=20the=20hook=20judges=20outcomes=20rathe?= =?UTF-8?q?r=20than=20invocations.=20Goal:=20Change=20engine/hooks/gh-writ?= =?UTF-8?q?e-verification/detect.py=20so=20=5Fproves=5Flanding=20requires?= =?UTF-8?q?=20the=20paired=20tool=20result=20to=20report=20OK,=20and=20add?= =?UTF-8?q?=20tests=20for=20the=20FAIL=20and=20UNCHECKED=20cases.=20Motiva?= =?UTF-8?q?tion:=20The=20hook's=20own=20purpose=20is=20to=20stop=20a=20mer?= =?UTF-8?q?ge=20claim=20that=20outran=20its=20landing=20check.=20Because?= =?UTF-8?q?=20it=20reads=20only=20the=20command=20text,=20a=20check=20that?= =?UTF-8?q?=20ran=20and=20reported=20failure=20clears=20it=20exactly=20as?= =?UTF-8?q?=20well=20as=20one=20that=20reported=20success,=20so=20the=20fa?= =?UTF-8?q?ilure=20mode=20it=20exists=20to=20catch=20is=20the=20one=20it?= =?UTF-8?q?=20cannot=20see.=20Alternative=20considerations:=20Parsing=20th?= =?UTF-8?q?e=20exit=20code=20alone=20was=20rejected=20because=20the=20scri?= =?UTF-8?q?pt=20prints=20its=20verdict=20on=20stdout=20and=20an=20exit=20c?= =?UTF-8?q?ode=20is=20not=20available=20for=20every=20recorded=20command?= =?UTF-8?q?=20shape.=20Leaving=20the=20detector=20as-is=20and=20adding=20a?= =?UTF-8?q?=20prose=20rule=20was=20rejected:=20the=20detector=20is=20the?= =?UTF-8?q?=20mechanism,=20so=20prose=20would=20be=20a=20second=20copy=20o?= =?UTF-8?q?f=20the=20same=20rule=20with=20no=20enforcement.=20Implementati?= =?UTF-8?q?on=20details:=20Extend=20bash=5Fcommands=5Fthis=5Fturn=20(detec?= =?UTF-8?q?t.py=20lines=20369-391)=20to=20carry=20each=20command's=20paire?= =?UTF-8?q?d=20tool=20result=20alongside=20its=20text=20rather=20than=20di?= =?UTF-8?q?scarding=20non-assistant=20records.=20Change=20=5Fproves=5Fland?= =?UTF-8?q?ing=20(detect.py=20lines=20291-302)=20to=20require=20an=20OK=20?= =?UTF-8?q?marker=20in=20that=20result,=20treating=20a=20missing=20result?= =?UTF-8?q?=20as=20not=20proving=20landing.=20Update=20merges=5Fmissing=5F?= =?UTF-8?q?landing=5Fproof=20(detect.py=20line=20306)=20to=20pass=20the=20?= =?UTF-8?q?results=20through.=20Add=20tests=20asserting=20a=20FAIL=20resul?= =?UTF-8?q?t=20and=20an=20UNCHECKED=20result=20both=20still=20flag,=20and?= =?UTF-8?q?=20that=20an=20OK=20result=20stays=20silent.=20Non-goals:=20Doe?= =?UTF-8?q?s=20not=20change=20the=20verifier=20script's=20exit=20contract,?= =?UTF-8?q?=20does=20not=20change=20which=20commands=20count=20as=20merges?= =?UTF-8?q?,=20does=20not=20widen=20the=20hook=20to=20other=20tools,=20and?= =?UTF-8?q?=20does=20not=20touch=20any=20skill=20prose.=20Layer:=20domain?= =?UTF-8?q?=20Feature=20state:=20active=20Files:=20engine/hooks/gh-write-v?= =?UTF-8?q?erification/detect.py,=20engine/hooks/gh-write-verification/tes?= =?UTF-8?q?ts/test=5Fhooks.py=20Change=20types:=20-=20engine/hooks/gh-writ?= =?UTF-8?q?e-verification/detect.py:=20modify=20-=20engine/hooks/gh-write-?= =?UTF-8?q?verification/tests/test=5Fhooks.py:=20modify=20Acceptance=20cri?= =?UTF-8?q?teria:=20-=20`python3=20-m=20unittest=20discover=20-s=20engine/?= =?UTF-8?q?hooks/gh-write-verification/tests=20-v`=20exits=200.=20-=20A=20?= =?UTF-8?q?landing=20check=20whose=20result=20printed=20FAIL=20still=20fla?= =?UTF-8?q?gs=20the=20merge.=20-=20A=20landing=20check=20whose=20result=20?= =?UTF-8?q?printed=20UNCHECKED=20still=20flags=20the=20merge.=20-=20A=20la?= =?UTF-8?q?nding=20check=20whose=20result=20printed=20OK=20stays=20silent.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Solution: Make the landing-proof detector require the check's OK result, not merely its invocation. Review claim: A landing check that printed FAIL or UNCHECKED does not satisfy the landing-proof hook. Review lane: behavior Safety invariant: No merge shape that is silent today becomes noisy: a merge followed by a landing check printing OK stays silent, and only FAIL, UNCHECKED, or a missing result newly flags. Effectiveness measurement: Three fixtures: a landing check printing OK stays silent, one printing FAIL flags, one printing UNCHECKED flags. The current detector passes only the first, so the suite fails before the change and passes after. Slice rationale: One detector and its tests in one hook directory; shares no file with the sweep-skill chain, so it runs independently rather than waiting on it. Architectural effect: The detector gains access to each command's paired tool result, so the hook judges outcomes rather than invocations. Goal: Change engine/hooks/gh-write-verification/detect.py so _proves_landing requires the paired tool result to report OK, and add tests for the FAIL and UNCHECKED cases. Motivation: The hook's own purpose is to stop a merge claim that outran its landing check. Because it reads only the command text, a check that ran and reported failure clears it exactly as well as one that reported success, so the failure mode it exists to catch is the one it cannot see. Alternative considerations: Parsing the exit code alone was rejected because the script prints its verdict on stdout and an exit code is not available for every recorded command shape. Leaving the detector as-is and adding a prose rule was rejected: the detector is the mechanism, so prose would be a second copy of the same rule with no enforcement. Implementation details: Extend bash_commands_this_turn (detect.py lines 369-391) to carry each command's paired tool result alongside its text rather than discarding non-assistant records. Change _proves_landing (detect.py lines 291-302) to require an OK marker in that result, treating a missing result as not proving landing. Update merges_missing_landing_proof (detect.py line 306) to pass the results through. Add tests asserting a FAIL result and an UNCHECKED result both still flag, and that an OK result stays silent. Non-goals: Does not change the verifier script's exit contract, does not change which commands count as merges, does not widen the hook to other tools, and does not touch any skill prose. Layer: domain Feature state: active Files: engine/hooks/gh-write-verification/detect.py, engine/hooks/gh-write-verification/tests/test_hooks.py Change types: - engine/hooks/gh-write-verification/detect.py: modify - engine/hooks/gh-write-verification/tests/test_hooks.py: modify Acceptance criteria: - `python3 -m unittest discover -s engine/hooks/gh-write-verification/tests -v` exits 0. - A landing check whose result printed FAIL still flags the merge. - A landing check whose result printed UNCHECKED still flags the merge. - A landing check whose result printed OK stays silent. Invoker-Finalize-Id: 275f6067-457a-4dde-8dbf-8542a8fff971 --- engine/hooks/gh-write-verification/detect.py | 66 ++++++++++++++---- .../gh-write-verification/tests/test_hooks.py | 68 +++++++++++++++++-- 2 files changed, 116 insertions(+), 18 deletions(-) diff --git a/engine/hooks/gh-write-verification/detect.py b/engine/hooks/gh-write-verification/detect.py index 47b73b81..7a287210 100644 --- a/engine/hooks/gh-write-verification/detect.py +++ b/engine/hooks/gh-write-verification/detect.py @@ -213,6 +213,7 @@ def self_match_message(hits: list[str]) -> str: r"|\bgit\s+branch\s+-r\s+--contains\b" r"|\bgit\s+branch\s+--contains\b[^\n]*\s-r\b" ) +LANDING_OK_RE = re.compile(r"(?m)^\s*OK:") VERIFY_SCRIPT_RELPATH = "gh-write-verification/verify_pr_landed_on_trunk.sh" UNVERIFIED_MERGE_MESSAGE = ( @@ -288,22 +289,33 @@ def silenced_mutations(raw_text: str) -> list[str]: return hits -def _proves_landing(command: str, number: str | None) -> bool: +CommandRecord = tuple[str, str | None] + + +def _proves_landing(command: str, number: str | None, result: str | None) -> bool: """True when this command checks where a merge commit actually landed. An invocation of the shipped verification script must name the PR it is - vouching for; a hand-rolled ancestry check is accepted as written, since - it takes a commit sha rather than a PR number. + vouching for, and the paired tool result must report the passing verdict. """ command = command or "" if not LANDING_PROOF_RE.search(command): return False if "verify_pr_landed_on_trunk" in command and number is not None: - return number in command - return True + if number not in command: + return False + return bool(result and LANDING_OK_RE.search(result)) + + +def _command_text(record: str | CommandRecord) -> str: + return record[0] if isinstance(record, tuple) else record + + +def _command_result(record: str | CommandRecord) -> str | None: + return record[1] if isinstance(record, tuple) else None -def merges_missing_landing_proof(commands: list[str]) -> list[str]: +def merges_missing_landing_proof(commands: list[str | CommandRecord]) -> list[str]: """PR subjects merged in this turn with no landing check run afterwards. Returns the merged subjects (a PR number, or "the current branch's PR" @@ -312,13 +324,17 @@ def merges_missing_landing_proof(commands: list[str]) -> list[str]: proof ran after the merge. """ subjects: list[str] = [] - for index, command in enumerate(commands): + for index, record in enumerate(commands): + command = _command_text(record) match = GH_PR_MERGE_RE.search(command or "") if not match: continue number = match.group("number") subject = f"PR #{number}" if number else "the current branch's PR" - if any(_proves_landing(later, number) for later in commands[index + 1:]): + if any( + _proves_landing(_command_text(later), number, _command_result(later)) + for later in commands[index + 1:] + ): continue if subject not in subjects: subjects.append(subject) @@ -359,6 +375,17 @@ def _text_content(data: dict) -> str: return "" +def _tool_result_text(block: dict) -> str: + content = block.get("content") + if isinstance(content, str): + return content + if isinstance(content, list): + return "\n".join( + b.get("text", "") for b in content if isinstance(b, dict) and b.get("type") == "text" + ) + return "" + + def _is_human_user_line(data: dict) -> bool: if data.get("type") != "user": return False @@ -366,7 +393,7 @@ def _is_human_user_line(data: dict) -> bool: return bool(text.strip()) and not text.lstrip().startswith("<") -def bash_commands_this_turn(raw_lines) -> list[str]: +def bash_commands_this_turn(raw_lines) -> list[CommandRecord]: """Bash tool commands issued since the last authored user message.""" parsed: list[dict] = [] for raw in raw_lines: @@ -380,8 +407,21 @@ def bash_commands_this_turn(raw_lines) -> list[str]: for index, data in enumerate(parsed): if _is_human_user_line(data): turn_start = index - commands: list[str] = [] - for data in parsed[turn_start:]: + records = parsed[turn_start:] + results: dict[str, str] = {} + for data in records: + message = data.get("message") + content = message.get("content") if isinstance(message, dict) else None + if not isinstance(content, list): + continue + for block in content: + if not isinstance(block, dict) or block.get("type") != "tool_result": + continue + tool_id = block.get("tool_use_id") + if tool_id: + results[str(tool_id)] = _tool_result_text(block) + commands: list[CommandRecord] = [] + for data in records: if data.get("type") != "assistant": continue message = data.get("message") @@ -395,7 +435,9 @@ def bash_commands_this_turn(raw_lines) -> list[str]: continue tool_input = block.get("input") if isinstance(tool_input, dict): - commands.append(str(tool_input.get("command") or "")) + tool_id = block.get("id") or block.get("tool_use_id") + result = results.get(str(tool_id)) if tool_id else None + commands.append((str(tool_input.get("command") or ""), result)) return commands diff --git a/engine/hooks/gh-write-verification/tests/test_hooks.py b/engine/hooks/gh-write-verification/tests/test_hooks.py index c329b15a..3e6a7f6e 100644 --- a/engine/hooks/gh-write-verification/tests/test_hooks.py +++ b/engine/hooks/gh-write-verification/tests/test_hooks.py @@ -57,15 +57,24 @@ def bash_payload(command: str) -> dict: return {"tool_name": "Bash", "cwd": HOOK_DIR, "tool_input": {"command": command}} -def transcript(commands: list[str]) -> str: +def transcript(commands: list[str | tuple[str, str | None]]) -> str: lines = [json.dumps({"type": "user", "message": {"role": "user", "content": "land the stack"}})] - for command in commands: + for index, item in enumerate(commands): + command, result = item if isinstance(item, tuple) else (item, None) + tool_id = f"bash-{index}" lines.append(json.dumps({ "type": "assistant", "message": {"content": [ - {"type": "tool_use", "name": "Bash", "input": {"command": command}} + {"type": "tool_use", "id": tool_id, "name": "Bash", "input": {"command": command}} ]}, })) + if result is not None: + lines.append(json.dumps({ + "type": "user", + "message": {"content": [ + {"type": "tool_result", "tool_use_id": tool_id, "content": result} + ]}, + })) handle = tempfile.NamedTemporaryFile("w", suffix=".jsonl", delete=False) handle.write("\n".join(lines) + "\n") handle.close() @@ -228,6 +237,9 @@ def test_entrypoint_denies_the_incident_wait_loop(self): class TestUnverifiedLanding(unittest.TestCase): + def proven(self, command: str) -> tuple[str, str]: + return (command, "OK: abc123 is an ancestor of origin/main\n") + def test_a_merge_with_no_landing_check_is_flagged(self): self.assertEqual( merges_missing_landing_proof(["gh pr merge 291 --squash --admin", "gh pr view 291"]), @@ -238,7 +250,7 @@ def test_every_unproven_merge_in_the_turn_is_flagged(self): commands = [ "gh pr merge 291 --squash", "gh pr merge 292 --squash", - "bash verify_pr_landed_on_trunk.sh 292", + self.proven("bash verify_pr_landed_on_trunk.sh 292"), ] self.assertEqual(merges_missing_landing_proof(commands), ["PR #291"]) @@ -251,9 +263,18 @@ def test_a_verified_landing_stays_silent(self): "git branch -r --contains 314f0447", ): self.assertEqual( - merges_missing_landing_proof(["gh pr merge 291 --squash", proof]), [], proof + merges_missing_landing_proof(["gh pr merge 291 --squash", self.proven(proof)]), [], proof ) + def test_a_landing_check_without_a_result_is_flagged(self): + self.assertEqual( + merges_missing_landing_proof([ + "gh pr merge 291 --squash", + "bash verify_pr_landed_on_trunk.sh 291", + ]), + ["PR #291"], + ) + def test_a_turn_with_no_merge_stays_silent(self): self.assertEqual(merges_missing_landing_proof(["git status", "gh pr view 291"]), []) @@ -271,7 +292,11 @@ def test_stop_entrypoint_denies_an_unproven_merge(self): def test_stop_entrypoint_allows_a_proven_merge(self): path = transcript([ "gh pr merge 291 --squash --admin", - 'bash "$HOME/.claude/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh" 291', + ( + 'bash "$HOME/.claude/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh" 291', + "pr=#291 repo=acme/widgets merged=true base=main merge_commit=abc123\n" + "OK: abc123 is an ancestor of origin/main\n", + ), ]) try: result = run_entrypoint(STOP_CHECK, {"transcript_path": path}) @@ -279,6 +304,37 @@ def test_stop_entrypoint_allows_a_proven_merge(self): finally: os.unlink(path) + def test_stop_entrypoint_denies_a_failed_landing_check(self): + path = transcript([ + "gh pr merge 291 --squash --admin", + ( + 'bash "$HOME/.claude/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh" 291', + "pr=#291 repo=acme/widgets merged=true base=stack merge_commit=abc123\n" + "FAIL: PR #291 in acme/widgets reports MERGED but abc123 is not on origin/main\n", + ), + ]) + try: + result = run_entrypoint(STOP_CHECK, {"transcript_path": path}) + self.assertEqual(result.returncode, 2, result.stderr) + self.assertIn("PR #291", result.stderr) + finally: + os.unlink(path) + + def test_stop_entrypoint_denies_an_unchecked_landing_check(self): + path = transcript([ + "gh pr merge 291 --squash --admin", + ( + 'bash "$HOME/.claude/hooks/gh-write-verification/verify_pr_landed_on_trunk.sh" 291', + "UNCHECKED: gh cannot resolve a repository here\n", + ), + ]) + try: + result = run_entrypoint(STOP_CHECK, {"transcript_path": path}) + self.assertEqual(result.returncode, 2, result.stderr) + self.assertIn("PR #291", result.stderr) + finally: + os.unlink(path) + def test_a_missing_transcript_fails_open(self): self.assertIsNone(decide_stop({"transcript_path": "/nonexistent/transcript.jsonl"})) self.assertIsNone(decide_stop({})) From da615bced1565ceb53b0cdd91a5db93146987c23 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:09:28 +0000 Subject: [PATCH 2/3] =?UTF-8?q?invoker:=20wf-1789236146378-22/verify-landi?= =?UTF-8?q?ng-detector-result=20=E2=80=94=20Run=20the=20deterministic=20pr?= =?UTF-8?q?oof=20for=20the=20landing=20detector's=20result=20reading.=20Re?= =?UTF-8?q?view=20claim:=20The=20hook's=20tests=20assert=20a=20failed=20la?= =?UTF-8?q?nding=20check=20still=20flags=20the=20merge.=20Review=20lane:?= =?UTF-8?q?=20proof=20Safety=20invariant:=20Proof-only;=20adds=20no=20prod?= =?UTF-8?q?uct=20behavior.=20Effectiveness=20measurement:=20The=20suite=20?= =?UTF-8?q?fails=20if=20a=20FAIL=20or=20UNCHECKED=20landing=20result=20cle?= =?UTF-8?q?ars=20the=20hook.=20Slice=20rationale:=20One=20proof=20slice=20?= =?UTF-8?q?for=20this=20step.=20Architectural=20effect:=20None;=20verifica?= =?UTF-8?q?tion=20only.=20Goal:=20Prove=20the=20landing=20detector's=20res?= =?UTF-8?q?ult=20reading=20deterministically.=20Motivation:=20The=20defect?= =?UTF-8?q?=20is=20a=20detector=20that=20reads=20the=20wrong=20input,=20so?= =?UTF-8?q?=20the=20proof=20drives=20it=20with=20results=20rather=20than?= =?UTF-8?q?=20commands.=20Alternative=20considerations:=20Manual=20verific?= =?UTF-8?q?ation=20was=20rejected=20as=20non-deterministic.=20Implementati?= =?UTF-8?q?on=20details:=20Execute=20the=20command=20below=20as=20the=20te?= =?UTF-8?q?rminal=20proof.=20Non-goals:=20No=20product=20edits=20here;=20p?= =?UTF-8?q?roof=20only.=20Layer:=20app=5Fregression=20Feature=20state:=20a?= =?UTF-8?q?ctive?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 1e117326-8ae6-4b5c-9862-bf9319098010 From ff276831d146e0b26b145e066ef7498109e6e826 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:10:20 +0000 Subject: [PATCH 3/3] =?UTF-8?q?invoker:=20wf-1789236146378-22/scrub-handof?= =?UTF-8?q?f-artifacts=20=E2=80=94=20Terminal=20check=20that=20no=20epheme?= =?UTF-8?q?ral=20inter-task=20handoff=20files=20remain.=20Review=20claim:?= =?UTF-8?q?=20The=20workflow=20leaves=20no=20scratch=20handoff=20artifacts?= =?UTF-8?q?=20behind.=20Review=20lane:=20cleanup=20Safety=20invariant:=20R?= =?UTF-8?q?ead-only;=20never=20deletes=20files,=20alters=20the=20index,=20?= =?UTF-8?q?or=20commits=20caller=20work.=20Effectiveness=20measurement:=20?= =?UTF-8?q?The=20gate=20fails=20when=20a=20plans/invoker-handoff.*=20or=20?= =?UTF-8?q?lens-*.json=20file=20is=20still=20present=20after=20the=20leaf?= =?UTF-8?q?=20tasks=20complete.=20Slice=20rationale:=20One=20terminal=20hy?= =?UTF-8?q?giene=20gate=20for=20the=20workflow.=20Architectural=20effect:?= =?UTF-8?q?=20None;=20check=20only.=20Goal:=20Confirm=20no=20ephemeral=20h?= =?UTF-8?q?andoff=20files=20survive=20the=20run.=20Motivation:=20Inter-tas?= =?UTF-8?q?k=20scratch=20files=20leak=20into=20diffs=20and=20read=20as=20p?= =?UTF-8?q?art=20of=20the=20work.=20Alternative=20considerations:=20Deleti?= =?UTF-8?q?ng=20them=20automatically=20was=20rejected;=20the=20gate=20repo?= =?UTF-8?q?rts,=20it=20does=20not=20mutate.=20Implementation=20details:=20?= =?UTF-8?q?Run=20scripts/scrub-handoff-artifacts.sh=20without=20--apply.?= =?UTF-8?q?=20Non-goals:=20No=20deletion,=20no=20index=20changes,=20no=20c?= =?UTF-8?q?ommits.=20Layer:=20app=5Fregression=20Feature=20state:=20active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 657b1e05-5809-43b8-a2ea-e4ef235c90a4