Excuses (6) handing the user a command needs an attempt first - #556
Open
EdbertChan wants to merge 1 commit into
Conversation
…tried A background-judged Stop hook. When a reply asks the user to run a command or do a step themselves, it checks the turn for an earlier attempt, and stays silent after a permission denial, a sandbox or classifier refusal, or a human-only step (password, OAuth consent, hardware). A judge result that could not be produced is reported as unchecked, never as clean. Built by Invoker workflow wf-1789281140471-18; its only failing check (check_install_effective.py) reads this machine's install and fails on clean origin/main as well. The cat-mode prose pointer is left for a follow-up PR so this one stays a single engine-runtime review unit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016VgzyERsebhUHizFhFqP2t Change-Id: I8fd1c26cf5fa020719f7f9452aa68acdacf2d259
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Small checker scripts read each reply Claude writes. This adds one that notices when Claude asks the person to run something Claude never tried.
The problem: agents hand work back too early. One wrote "please run" a setup step without trying it, and the person answered "why don't you just run it for me".
The cause: the rule against this lived only in written guidance. Nothing read replies for it, so it kept happening.
The fix: a new checker asks a small model whether the reply hands work back with no earlier attempt. The answer arrives on the next turn.
Review Claim
A reply that asks the person to do a step the agent never attempted is flagged on the next turn, unless a refusal came first or only a human can do the step.
Review Lane
policy
Review Unit
engine-runtime
Safety Invariant
A hand-back that follows a permission denial, a sandbox or classifier refusal, or a step only a human can do (password, OAuth consent, hardware) is never flagged. If the judge cannot answer, the next turn says it could not judge; it never reports clean.
Slice Rationale
One new hook with its phrase dictionary, tests, and install wiring. The one-line pointer from the cat-mode hand-back rule to this hook is a separate corpus-lesson PR, because it is a different review unit.
Non-goals
Test Plan
Test Plan
Built by Invoker workflow wf-1789281140471-18. Its only failing check,
python3 scripts/check_install_effective.py, reads this machine's installed hooks rather than the branch, and fails on a clean origin/main checkout too (exit 1, 116 "bypasses the metrics runner" lines, 0 mentioning this hook).Run on this branch:
python3 -m unittest discover -s engine/hooks/handback-needs-attempt/tests→ OKpython3 scripts/check_hook_test_coverage.py→ OK (38 hooks)python3 scripts/check_no_silent_hook_except.py→ checked 205 filespython3 scripts/check_rules_are_wired.py→ OK (9 checkers)python3 scripts/check_no_new_comments.py --base origin/main→ okpython3 engine/skills/make-pr/scripts/preflight.py --base origin/main --body-file <this body>→ see belowRevert Plan
Revert Plan
git revert <merge sha>./install.shso the hook link and settings entry are removed🤖 Generated with Claude Code
https://claude.ai/code/session_016VgzyERsebhUHizFhFqP2t