fix(commands): stop teaching the discarded-stderr shape, and gate it (v1.40.0) - #58
fix(commands): stop teaching the discarded-stderr shape, and gate it (v1.40.0)#58lapc506 wants to merge 2 commits into
Conversation
…(v1.40.0)
The toolkit ships a discard-stderr hook that refuses a command an agent RUNS.
That hook has no reach at all over the same shape written INTO a command or a
skill -- and those files exist to be copied and executed. So the toolkit was
blocking what its own commands taught.
Measured across the tree: 164 discard sites, 152 in a refused form. Of those,
34 sat in commands/, skills/ and agents/ -- the instructing surface -- telling
the reader to write exactly what the hook refuses.
A discarded stream makes a FAILING command indistinguishable from a SUCCEEDING
one that printed nothing, and the empty result then reads as "none found"
rather than "it errored".
All 34 now append stderr to ${MNM_LOG:-/tmp/make-no-mistakes.log}, a form that
is self-contained so no site depends on a variable an earlier step was supposed
to define. Two files were left alone deliberately: commands/parallelize.md
STATES the rule in prose, and skills/merge-advisor/SKILL.md uses the legal
reverse order.
THE MECHANISM, because a sweep with no gate undoes itself:
scripts/check-discard-stderr.mjs, wired into npm run check-discard-stderr and
into prepublishOnly, and asserted by the suite -- not a gate that nothing runs.
Mention is not execution, and that is the hard part. In markdown only FENCED
blocks are read as commands; inline backticks are prose, because a rule that
says "never write X" has to be able to write X. Not hypothetical: while
building this, a sibling guard blocked the command that was COUNTING the
violations, and separately blocked a PR body that merely QUOTED a redirect
while explaining why its order is the safe one. A guard that refuses ordinary
work gets bypassed, and a bypassed guard carries no information at all.
Order is the whole distinction, and the suite's central assertion is that the
two spellings sharing a token set get opposite verdicts: duplicating stderr to
the original stdout BEFORE stdout is retargeted survives and is allowed;
retargeting stdout first loses both and is refused. The bare stdout form is
refused too -- stderr survives it, which is not the whole harm, because it
throws away the ANSWER along with the noise.
Scope is split on purpose. commands/ skills/ agents/ is a hard gate at zero.
hooks/ scripts/ is real shell with 113 remaining sites, counted and reported
but NOT enforced: a gate that reds the repo it guards is one that gets deleted.
Making that half a ratchet on added lines is named as intended, not as existing.
An unreadable file is a finding rather than a skip, so a permissions problem
cannot read as a passing scan -- the same collapse the checker exists to
prevent, one level up.
Tests 72/72, 12 new, one negative control per refused form.
Created by Claude Code on behalf of @lapc506
…er-repo name Two corrections in one, both caught in review. 1. THE PATTERN IS NOT DEFINED IN THE CHECKER ANY MORE. The first version carried its own regex for what counts as a discard. The toolkit already defines that in hooks/rules/rules.yaml -- the SSoT the hook uses, with rules.json generated from it and CI failing on drift. So there were two implementations of one measurement, and a second implementation drifts: the rule tightens in one place and the other keeps passing what it started refusing. Silently. Which is precisely the class of defect this checker exists to catch, committed inside the checker. loadRule() reads rules.json and extracts the discard-stderr pattern and its mention exemption. The only transformation is a POSIX bracket class to its JS equivalent, exported as a named function so it can be asserted rather than hidden in an inline replace. A missing rule THROWS rather than falling back to a private pattern -- a checker that substituted its own copy would report PASS while checking something nobody approved -- and exits 2 (could-not-determine), never 1 (found-violations). What this file still owns is which LINES of a document are commands, which the rule genuinely cannot answer: a hook receives one command string, this receives a file. Proven by mutation, because a de-duplication that is claimed and not measured is just a comment: loosening the SHARED rule breaks 6 tests (the checker really does inherit), restoring a private copy in the checker breaks 7, unmutated control 17/17. 2. NO NEW CONSUMER-REPO NAMES IN A PUBLIC-SOURCE PLUGIN. Three comments I added named a specific consuming repository to illustrate a policy difference. This plugin is BSL-1.1 public-source and build-rules.mjs carries an IP-leak guard whose own header says client/org names do not belong anywhere in committed code. Writing them was an error regardless of whether that particular name sits in the gitignored forbidden list -- measured: it does not, since rules.yaml already contains it and the guard passes. All three now say "a consuming repo may hold a stricter local policy", which is the portable statement and the more accurate one for a general-purpose plugin. Separately findable and NOT fixed here: that guard scans only the serialized rules, so scripts/, src/, commands/ and CHANGELOG.md are outside its reach while its header promises "anywhere in committed code". Suite 77/77. build-rules regenerates rules.json with no drift. Created by Claude Code on behalf of @lapc506
There was a problem hiding this comment.
✅ Approved
Approved — 0 blockers, 1 P3. Confidence: 4.80/5.00.
Walkthrough
⚠️ Governance Notice: This PR targets themainbranch directly, bypassing the standard GitFlow pipeline (feature→develop→main). While the implementation itself is robust and correct, please ensure this direct merge tomainis intended or whether it should be retargeted todevelop.
Walkthrough
This PR addresses an inconsistency where the toolkit blocked agents from running commands with discarded standard error (2>/dev/null) while its own instructing surface (commands/, skills/, and agents/) taught the bad practice. It performs a comprehensive sweep of 34 instances, replacing them with safe redirection to MNM_LOG (defaulting to /tmp/make-no-mistakes.log), and introduces a strict build-time and pre-publish verification script (scripts/check-discard-stderr.mjs) to permanently prevent future regressions.
Areas Reviewed
- Instructing Surface: Markdown files in
commands/,skills/, andagents/to verify redirection correctness. - Checker Mechanism:
scripts/check-discard-stderr.mjsto evaluate how commands are isolated from prose in markdown, and how rules are loaded dynamically fromrules.json. - Testing:
src/audit/discard-stderr.test.tsto ensure coverage of both regex translation and document parsing logic. - Configuration: Integration of the checker in
package.json(prepublishOnlyand standard scripts).
Safety Rationale
The PR is safe to merge because it eliminates silent command execution errors without changing functional code, introduces a local test harness that passes perfectly (77/77), and separates gating (strictly enforced for user instructions) from ratcheting (reported but not blocking for internal shell scripts).
Verdict
Approved — 0 blockers, 1 P3.
🔵 P3 — Minor
src/audit/discard-stderr.test.ts:53— 🔵 P3 (minor) — ExecutingexecFileSync('cat', ...)makes the test suite dependent on the Unixcatutility. This utility is not natively available on Windows environments (without WSL/Git Bash), which causes test failures for developers working on Windows. Using Node's nativereadFileSyncinstead makes the test fully cross-platform.
[pass 1]
Total findings: 1 business context (1 total)
| // rule, this checker tightens with it; if someone reintroduces a private | ||
| // copy here, this fails. | ||
| const raw = JSON.parse( | ||
| execFileSync('cat', ['hooks/rules/rules.json'], { encoding: 'utf8' }), |
There was a problem hiding this comment.
🔵 P3 (minor) — Executing execFileSync('cat', ...) makes the test suite dependent on the Unix cat utility. This utility is not natively available on Windows environments (without WSL/Git Bash), which causes test failures for developers working on Windows. Using Node's native readFileSync instead makes the test fully cross-platform.
[pass 1]
The toolkit was blocking what its own commands taught
It ships a
discard-stderrrule that refuses a command an agent runs. That rule has no reach at all over the same shape written into a command or a skill — and those files exist to be copied and executed.Measured across the tree:
commands/skills/agents/— the instructing surfaceA discarded stream makes a failing command indistinguishable from a succeeding one that printed nothing, and the empty result then reads as "none found" rather than "it errored".
The sweep
All 34 now append stderr to
${MNM_LOG:-/tmp/make-no-mistakes.log}— self-contained, so no site depends on a variable some earlier step was supposed to define.Two files were left alone deliberately, and neither is an oversight:
commands/parallelize.mdstates the rule in prose. It already told readers not to do this.skills/merge-advisor/SKILL.mduses the legal reverse order.The mechanism, because a sweep with no gate undoes itself
scripts/check-discard-stderr.mjs, wired intonpm run check-discard-stderrandprepublishOnly, and asserted by the suite. Not a gate that nothing runs.The pattern is NOT defined in the checker
It is read from the shared
discard-stderrrule inhooks/rules/rules.yaml, via therules.jsonthe build already generates and CI already guards against drift.The first draft carried its own copy of the regex — a second implementation of one measurement. That drifts: the rule tightens in one place and the other keeps passing what it started refusing, silently, which is the exact class of defect this checker exists to catch, committed inside the checker. Adding a form now means editing
rules.yaml; both consumers inherit it and neither can disagree with the other.The only transformation is a POSIX bracket class to its JS equivalent, exported as a named function so it can be asserted rather than hidden in an inline replace. A missing rule throws rather than falling back to a private pattern, and exits 2 (could-not-determine), never 1 (found-violations) — a checker that substituted its own copy would report PASS while checking something nobody approved.
What the checker does own: which lines are commands
The rule genuinely cannot answer that — a hook receives one command string, this receives a document.
In markdown only fenced blocks count. Inline backticks are prose, because a rule that says "never write X" has to be able to write X. Not hypothetical: building this, a sibling guard blocked the command that was counting the violations, and separately blocked a PR body that merely quoted a redirect while explaining why its order is the safe one. Four such blocks across the session, every one reported rather than routed around and never with a marker — a guard that refuses ordinary work gets bypassed, and a bypassed guard carries no information at all.
Outside markdown every line is a command, comments included: a comment demonstrating the bad form is still the line a reader copies.
Order is the whole distinction
The suite's central assertion is that the two spellings sharing a token set get opposite verdicts: duplicating stderr to the original stdout before stdout is retargeted survives and is unmatched by the rule; retargeting stdout first loses both and is matched.
The bare stdout form is not in this toolkit's rule. A consuming repo may hold a stricter local policy, and a test pins that divergence so adopting it stays a deliberate edit to
rules.yaml— which changes the hook too — instead of a silent difference between two consumers.Scope is split on purpose
commands/skills/agents/hooks/scripts/A gate that reds the repo it guards is one that gets deleted. Turning the second half into a ratchet on added lines is the next step, named here as intended, not as existing.
An unreadable file is reported as a finding rather than skipped, so a permissions problem cannot read as a passing scan — the same collapse the checker exists to prevent, one level up.
Verification
npm run check-discard-stderr→ gate 0 findings, PASS, pattern sourced fromrules.json#discard-stderrnpx vitest run→ 77/77, 13 files (17 new)npm run build-rules→ regeneratesrules.jsonwith no driftA finding this PR does NOT fix
build-rules.mjscarries an IP-leak guard whose header says client/org names do not belong "anywhere in committed code". It scans only the serialized rules (build-rules.mjs:184-190), soscripts/,src/,commands/,skills/andCHANGELOG.mdare outside its reach entirely.Found the hard way: three comments added in this branch named a specific consuming repository, and nothing stopped them. They are gone — all three now say "a consuming repo may hold a stricter local policy", which is the portable statement and the more accurate one for a general-purpose plugin. Widening the guard is the rule owner's decision, not a side effect of this PR.
It is the same shape as the defect above: a control whose stated scope is wider than its mechanism.
Ordering note — a live example of the sibling PR
Cut from
mainand bumps to 1.40.0, assuming #57 (merge-advisor, 1.39.0) lands first. Both PRs touchpackage.json,plugin.json,marketplace.jsonandCHANGELOG.md.Neither conflicts with
maintoday; they conflict with each other. That is exactly the latent-conflict class #57's predicate 5 exists to surface, and it is invisible togh pr liston both. If this one merges first, #57 needs a rebase and a renumber to 1.40.0, and this becomes 1.39.0.Created by Claude Code on behalf of @lapc506