Excuses (6) handing the user a command needs an attempt first

docs/ecosystem.md

@@ -73,6 +73,7 @@ again.
 | `demo-freeze` | hook |
 | `explicit-failures` | hook (advisory; always on) |
 | `external-claim-gate` | hook (PreToolUse on `Bash`; blocks a gh issue/comment/release/api write whose body claims a cause or fix with no evidence; blocks as UNCHECKED when the body cannot be read) |
+| `handback-needs-attempt` | hook (Stop; background-judges user hand-backs that lacked an agent attempt) |
 | `playbook-router` | hook (UserPromptSubmit; injects the steps of the one playbook a prompt names) |
 | `diu-stop` | hook (Stop; blocks a reply over the word limit or with an unproven claim, and asks the background judge whether the reply used wording from its `phrases/` lists, waiting for that answer so a hit blocks the same turn) |
 | `frustration-watchdog` | hook |

engine/hooks/handback-needs-attempt/README.md

@@ -0,0 +1,75 @@
+# handback-needs-attempt
+
+On Claude `Stop`, flag a reply that hands the user a command or actionable
+step the agent could have attempted when the current turn has no attempt before
+the hand-back.
+
+This is the mechanical backstop for cat-mode's rule that a hand-back is an
+unverified claim. The hook does not decide that from local wording rules. On
+every Stop it hands the latest reply and turn exchange to the background judge
+using
+[`engine/hooks/llm-judge/phrases/handback-needs-attempt.json`](../llm-judge/phrases/handback-needs-attempt.json).
+A hit arrives on a later turn through the shared [`llm-judge`](../llm-judge/README.md)
+inbox and carries the dictionary's `on_hit` text. The live reply is never held
+up. If the judge result was unchecked, the inbox reports "could not judge"
+instead of treating the reply as clean. Fail-open.
+
+Stay silent after a permission denial, sandbox refusal, classifier refusal, or a
+step that only the human can do, such as entering a password, granting OAuth
+consent, approving a hardware prompt, or connecting hardware. Quoted or
+documented instructions are not hand-backs.
+
+When prompted, attempt the command or step first and report the result. If it
+truly requires the human, name the permission denial, sandbox or classifier
+refusal, password, OAuth consent, or hardware requirement that makes it
+human-only. `stop_hook_active` is a one-rewrite escape hatch.
+
+## Model-judged path
+
+`enqueue_judge` in `detect.py` reads the transcript, takes the current reply,
+builds a phrase-dictionary job from the latest exchange, and sends it to
+`llm-judge`. The dictionary defines the meaning with `match` and `not_match`
+examples and supplies the static `on_hit` follow-up text.
+
+No job is sent when `stop_hook_active` is set, when this transcript or reply was
+already prompted, or when the reply is empty. Inside a judge run
+(`CATSTACK_LLM_JUDGE_CHILD=1`) `llm-judge` refuses the job.
+
+The model call runs in a detached background process, so the reply is never held
+up. Runners are tried in `llm-judge` order: `codex` (gpt-5.3-codex-spark), then
+`claude` (haiku, hooks off), then `cursor-agent`, first answer wins.
+
+The verdict reports one turn later. On the next prompt the `llm-judge` inbox
+shows a hit as the dictionary's `on_hit` text. If no runner could answer, or the
+result could not be checked, the inbox says "could not judge" instead of staying
+quiet. A clean verdict shows nothing.
+
+`llm-judge` is loaded from the sibling folder (`../llm-judge/judge.py`), which
+sits next to this one in the repo and in each harness's `hooks/` folder. If the
+hook input is malformed JSON, the wrapper returns quietly. If judge enqueue
+raises, the hook writes `catstack-hook-error handback-needs-attempt: <error>` to
+stderr and otherwise stays fail-open.
+
+To grow coverage, add the real text of any miss to the dictionary's `match`
+phrases, or the real text of any false alarm to `not_match`. Do not add a
+pattern to this hook; the prose meaning belongs in the phrase dictionary.
+
+## Files
+
+- `detect.py` - judge enqueue + once-per-transcript state
+- `claude_stop_check.py` - Claude `Stop` wrapper
+- `claude.hook.json` - Claude hook fragment
+- `install_claude_hook.py` - idempotent Claude settings merge
+- `tests/` - positive, exception, fail-open, install, and unchecked outcomes
+- `../llm-judge/phrases/handback-needs-attempt.json` - phrase dictionary
+
+## Install
+
+`./install.sh` from the repo root. Restart the harness.
+
+## Tests
+
+```sh
+python3 -m unittest discover -s engine/hooks/handback-needs-attempt/tests -v
+python3 scripts/check_hook_test_coverage.py engine/hooks/handback-needs-attempt
+```

engine/hooks/handback-needs-attempt/claude.hook.json

@@ -0,0 +1,16 @@
+{
+  "hooks": {
+    "Stop": [
+      {
+        "matcher": "*",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "python3 $HOME/.claude/hooks/handback-needs-attempt/claude_stop_check.py",
+            "timeout": 10
+          }
+        ]
+      }
+    ]
+  }
+}

engine/hooks/handback-needs-attempt/claude_stop_check.py

@@ -0,0 +1,17 @@
+#!/usr/bin/env python3
+import json
+import sys
+
+from detect import try_enqueue_judge
+
+
+def main() -> None:
+    try:
+        payload = json.load(sys.stdin)
+    except (json.JSONDecodeError, OSError):
+        return
+    try_enqueue_judge(payload if isinstance(payload, dict) else {})
+
+
+if __name__ == "__main__":
+    main()

engine/hooks/handback-needs-attempt/detect.py

@@ -0,0 +1,143 @@
+from __future__ import annotations
+
+import functools
+import hashlib
+import importlib.util
+import json
+import os
+import sys
+import uuid
+
+HOOKS_DIR = os.path.dirname(os.path.abspath(__file__))
+LLM_JUDGE_PATH = os.path.join(os.path.dirname(HOOKS_DIR), "llm-judge", "judge.py")
+PHRASES_PATH = os.path.join(os.path.dirname(HOOKS_DIR), "llm-judge", "phrases.py")
+STATE_DIR = os.environ.get(
+    "HANDBACK_NEEDS_ATTEMPT_STATE_DIR",
+    os.path.join(os.path.expanduser("~"), ".cache", "catstack-handback-needs-attempt"),
+)
+
+
+def _state_file(key: str) -> str:
+    digest = hashlib.sha1(key.encode()).hexdigest()[:16]
+    return os.path.join(STATE_DIR, f"{digest}.prompted")
+
+
+def already_prompted(key: str) -> bool:
+    return os.path.isfile(_state_file(key))
+
+
+def mark_prompted(key: str) -> None:
+    os.makedirs(STATE_DIR, exist_ok=True)
+    with open(_state_file(key), "w", encoding="utf-8") as handle:
+        handle.write(key + "\n")
+
+
+def _text(data: dict) -> str:
+    message = data.get("message")
+    content = message.get("content") if isinstance(message, dict) else data.get("content")
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        parts = []
+        for block in content:
+            if isinstance(block, str):
+                parts.append(block)
+            elif isinstance(block, dict):
+                if block.get("type") == "text":
+                    parts.append(block.get("text") or "")
+                elif block.get("type") in {"tool_use", "tool_result"}:
+                    parts.append(json.dumps(block, ensure_ascii=False, sort_keys=True))
+        return "\n".join(parts)
+    return ""
+
+
+def _human_user(data: dict) -> bool:
+    if data.get("type") != "user":
+        return False
+    message = data.get("message")
+    content = message.get("content") if isinstance(message, dict) else data.get("content")
+    if isinstance(content, str):
+        return bool(content.strip()) and not content.lstrip().startswith(("<command-", "<task-notification", "<system"))
+    return isinstance(content, list) and any(isinstance(block, dict) and block.get("type") == "text" for block in content)
+
+
+def resolve_transcript(payload: dict) -> str:
+    for key in ("agent_transcript_path", "transcript_path", "transcriptPath"):
+        value = payload.get(key)
+        if isinstance(value, str) and os.path.isfile(value):
+            return value
+    return ""
+
+
+def exchange_from_transcript(path: str) -> str:
+    if not path or not os.path.isfile(path):
+        return ""
+    try:
+        with open(path, encoding="utf-8") as handle:
+            lines = [json.loads(line) for line in handle if line.strip()]
+    except (OSError, json.JSONDecodeError):
+        return ""
+    start = 0
+    for index, data in enumerate(lines):
+        if isinstance(data, dict) and _human_user(data):
+            start = index
+    return "\n".join(_text(data) for data in lines[start:] if isinstance(data, dict))
+
+
+def last_assistant_text(payload: dict, path: str) -> str:
+    for key in ("last_assistant_message", "last-assistant-message", "lastAssistantMessage"):
+        value = payload.get(key)
+        if isinstance(value, str) and value.strip():
+            return value
+    exchange = exchange_from_transcript(path)
+    return exchange
+
+
+def decide(_payload: dict) -> None:
+    return None
+
+
+@functools.cache
+def _judge():
+    spec = importlib.util.spec_from_file_location("llm_judge", LLM_JUDGE_PATH)
+    if spec is None or spec.loader is None:
+        raise ImportError(LLM_JUDGE_PATH)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+@functools.cache
+def _phrases():
+    spec = importlib.util.spec_from_file_location("llm_judge_phrases", PHRASES_PATH)
+    if spec is None or spec.loader is None:
+        raise ImportError(PHRASES_PATH)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+def enqueue_judge(payload: dict) -> str | None:
+    if not isinstance(payload, dict) or payload.get("stop_hook_active"):
+        return None
+    path = resolve_transcript(payload)
+    reply = last_assistant_text(payload, path)
+    exchange = exchange_from_transcript(path)
+    text = "\n\nLATEST REPLY:\n" + reply + ("\n\nTURN EXCHANGE:\n" + exchange if exchange else "")
+    key = path or text[:200]
+    if not reply.strip() or already_prompted(key):
+        return None
+    dictionary = _phrases().load("handback-needs-attempt")
+    job = _phrases().job(dictionary, path, text)
+    job["id"] = uuid.uuid4().hex
+    job_id = _judge().enqueue(job)
+    if job_id is not None:
+        mark_prompted(key)
+    return job_id
+
+
+def try_enqueue_judge(payload: dict) -> None:
+    try:
+        enqueue_judge(payload)
+    except Exception as exc:
+        print(f"catstack-hook-error handback-needs-attempt: {type(exc).__name__}: {exc}", file=sys.stderr)

engine/hooks/handback-needs-attempt/install_claude_hook.py

@@ -0,0 +1,38 @@
+#!/usr/bin/env python3
+import json
+import os
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+SETTINGS_PATH = os.path.expanduser("~/.claude/settings.json")
+FRAGMENT_PATH = os.path.join(HERE, "claude.hook.json")
+MARKER = "handback-needs-attempt/claude_stop_check.py"
+EVENT = "Stop"
+
+
+def _is_ours(entry: dict) -> bool:
+    return any(MARKER in h.get("command", "") for h in entry.get("hooks", []))
+
+
+def merge_hook(settings: dict, fragment: dict) -> bool:
+    entries = settings.setdefault("hooks", {}).setdefault(EVENT, [])
+    incoming = fragment.get("hooks", {}).get(EVENT, [])
+    before = json.dumps(entries, sort_keys=True)
+    entries[:] = [entry for entry in entries if not _is_ours(entry)] + incoming
+    return json.dumps(entries, sort_keys=True) != before
+
+
+def main() -> None:
+    settings = {}
+    if os.path.exists(SETTINGS_PATH):
+        with open(SETTINGS_PATH, encoding="utf-8") as handle:
+            settings = json.load(handle)
+    with open(FRAGMENT_PATH, encoding="utf-8") as handle:
+        fragment = json.load(handle)
+    if not merge_hook(settings, fragment):
+        print("ok      claude Stop handback-needs-attempt already up to date")
+        return
+    os.makedirs(os.path.dirname(SETTINGS_PATH), exist_ok=True)
+    with open(SETTINGS_PATH, "w", encoding="utf-8") as handle:
+        json.dump(settings, handle, indent=2)
+        handle.write("\n")
+    print("link    claude Stop handback-needs-attempt merged into settings.json")

engine/hooks/handback-needs-attempt/tests/fixtures/handbacks.json

@@ -0,0 +1,6 @@
+[
+  {"name": "command handback", "reply": "Please run: invoker-cli setup slack ... then tell me when it completes", "match": true},
+  {"name": "xcode handback", "reply": "Simulator build still passes. Now on your end in Xcode:", "match": true},
+  {"name": "permission denial", "reply": "Permission to use Bash was denied, so please run the command yourself.", "match": false},
+  {"name": "oauth consent", "reply": "Please complete the OAuth consent screen on your end.", "match": false}
+]