Skip to content

fix: send the block report to stderr so the agent actually receives it - #65

Merged
ryanleecode merged 3 commits into
masterfrom
feature/block-report-to-stderr
Aug 25, 2026
Merged

fix: send the block report to stderr so the agent actually receives it#65
ryanleecode merged 3 commits into
masterfrom
feature/block-report-to-stderr

Conversation

@systemfsoftware-maker

Copy link
Copy Markdown
Collaborator

The defect

A flagged verdict exited 2 with its report on stdout. Exit 2 is the blocking code, and the PostToolUse contract forwards the hook's stderr to the model and discards stdout (hooks reference). The report was written to the one stream the host throws away, so the block arrived carrying no explanation of what was flagged — the model saw only that something blocked it.

Measured against a real host before the change: a Write payload adding // set x to 1 above const x = 1; produced a correct 611-byte report on stdout with exit 2 and an empty stderr, and the model received nothing from it. Where the hook was reached through a wrapper that emitted anything on stderr, that unrelated text became the block reason instead — which is how this presents as "the hook is broken".

The change

print!eprint! on the Block arm only, via two named emit paths so the stream choice is greppable rather than commented: emit_to_model (stderr, what a host forwards) and emit_to_debug_log (stdout, what it keeps). Exit codes are untouched; the pass note stays on stdout.

Why the existing tests missed it

clean_payload_exits_zero and flagged_payload_exits_with_the_blocked_contract pass both before and after this change. They pinned the exit code and never the stream — precisely the uncovered half of the contract that let this ship.

The new gate was itself reviewed adversarially, and its first version certified less than it looked like. Two mutations passed it:

Mutation Why it passed Now
eprint!("{input}") — dump the raw hook JSON the anchor was # TODO: fix this later, a string the payload supplies, so echoing the input satisfied it 4 failures
eprint!("# TODO: fix this later\n") — anchor only single contains with no structural assertion 4 failures

A check keyed on a value its own input provided certifies nothing. The anchors are now three strings only the report path can produce — the header, the action footer, and the classifier's reason text.

Coverage the gate was also missing, each verified by a mutation that used to pass and now fails:

  • the pass note's stream — moving it to stderr was previously invisible (1 failure)
  • Edit and MultiEdit — they share the Block arm with Write and had no binary-level test, so a per-tool branch could regress them unnoticed
  • --prompt — the substituted report travels the same arm

Docs

The npm page said the hook "prints" the report, which reads as stdout; the project README's demo redirected in a way that would now drop it. Both name the stream, and the exit-2 claim they rest on is cited rather than asserted.

Cargo.lock carries a one-line version sync — it still recorded 0.1.7 after the crate moved to 0.1.8, so any build regenerated it as an unrelated diff.

Verification

  • cargo fmt --check clean
  • cargo clippy --all-targets -- -D warnings — 0 findings
  • cargo test --all-targets — 101 passed, 0 failed
  • Falsified: with main.rs reverted to master, the stream tests fail while both pre-existing exit-code tests still pass
  • Four mutations killed: input-echo (4), anchor-only (4), stdout regression (4), pass-note stream (1)
  • Changeset validator OK; minor, because the entire observable effect of the binary changed for its primary consumer

The classifier is untouched, so no mutation run was required.

Not in scope

check.rs:40-58 never reads the file, so an Edit payload is judged as if the patch fragment were the file — fragment-offset line numbers, and a comment-only fragment is acquitted outright. tests/pipeline.rs:63-67 and :69-73 pin that acquittal as intended behaviour, so making it honest is a design change, not a bug fix, and it is left for a separate decision. This PR is orthogonal: it only changes which stream the verdict travels on.

A flagged verdict exited 2 with its report on stdout. Exit 2 is the
blocking code, and the PostToolUse contract forwards the hook's stderr to
the model and discards stdout, so the report was written to the one stream
the host throws away. The block landed with no explanation attached: the
model saw only that something blocked it, never which comment or why.

Measured against a real host before the change: the payload for a write
adding `// set x to 1` above `const x = 1;` produced a correct 611-byte
report on stdout with exit 2 and an empty stderr, and the model received
nothing from it. Where the hook was reached through a wrapper that emitted
anything at all on stderr, that unrelated text became the block reason
instead, which is how this presented as "the hook is broken".

`print!` becomes `eprint!` on the Block arm only. Exit codes are untouched,
and the pass note stays on stdout, where a host sends exit-0 stdout to its
debug log.

The two existing exit-code tests pass both before and after this change,
which is precisely why it shipped: they pinned the code and never the
stream. `flagged_payload_reports_on_stderr_only` now pins both halves —
report present on stderr, stdout empty — and it fails on the pre-change
binary (2 passed, 1 failed) and passes after.

Also syncs Cargo.lock, which still recorded 0.1.7 after the crate moved to
0.1.8, so any build regenerated it as an unrelated diff.

Verified: cargo fmt --check, cargo clippy --all-targets -D warnings (0
findings), cargo test --all-targets (97 passed, 0 failed).
The stream choice was carried by a comment above `eprint!`. Naming the two
emit paths carries it in code: `emit_to_model` is the stream a host forwards
on exit 2, `emit_to_debug_log` is the one it keeps to itself. The comment is
deleted rather than shortened, since a shortened restatement is still a
restatement.

Test failure messages lose the same prose for the same reason; the test
names already say which contract broke.
Review found the gate certified less than it appeared to. Two mutations
passed it:

- `eprint!("{input}")` — dumping the raw hook JSON. The anchor was
  `# TODO: fix this later`, a string the *payload* supplies, so echoing the
  input satisfied it. A check keyed on a value its own input provided
  certifies nothing.
- `eprint!("# TODO: fix this later\n")` — the anchor and nothing else. The
  model would get a bare comment with no file, no line, no reason.

The anchors are now three strings only the report path can produce: the
header, the action footer, and the classifier's reason text. Both mutations
now fail, as does the original stdout regression.

Coverage the gate was missing, each verified by a mutation that used to
pass and now fails:

- the pass note's stream — moving it to stderr was invisible before
- `Edit` and `MultiEdit`, which share the Block arm with `Write` and had no
  binary-level test, so a per-tool branch could regress them unnoticed
- `--prompt`, whose substituted report travels the same arm

Docs name the stream they were relying on: the npm page said "prints",
which reads as stdout, and the project README's demo redirected in a way
that would now drop the report. The exit-2 claim both rest on is cited
rather than asserted.

Verified: cargo fmt --check, clippy --all-targets -D warnings (0), cargo
test --all-targets (101 passed, 0 failed); four mutations killed
(input-echo 4 failures, anchor-only 4, stdout regression 4, pass-note 1);
changeset validator OK.
@ryanleecode
ryanleecode merged commit 3c1dec6 into master Aug 25, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants