From eaa95047932bf611b31df321db0069632060bb44 Mon Sep 17 00:00:00 2001 From: Edbert Chan Date: Sat, 12 Sep 2026 20:01:53 -0700 Subject: [PATCH 1/2] [Hooks Cannot See] (2) Require the output to entail the claim, not merely agree The session behind slice (1) ran one command against one container image and proved that image had no corepack. That was written up as corepack having been removed from Node 25+, a version-boundary claim no command in the session ever checked. The output was true and the sentence was wider than the output. prove-it already covers a missing check and a hedge. It did not cover a check that ran, came back narrower than the claim, and got filed under the wider heading anyway -- which is the common case, because the cheap check is always the reachable instance. It binds hardest in a correction: restating the original overclaim while pasting a narrower proof relabels the mistake as a fix, which is what happened here before the rule existed. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_011ZqCxSQVpmM9Rx37C7irgf Change-Id: Ibc32514be641d37277115faf6068e75ea02e0083 --- corpus/skills/principle-prove-it/SKILL.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/corpus/skills/principle-prove-it/SKILL.md b/corpus/skills/principle-prove-it/SKILL.md index 237048ff..48fffdff 100644 --- a/corpus/skills/principle-prove-it/SKILL.md +++ b/corpus/skills/principle-prove-it/SKILL.md @@ -34,6 +34,22 @@ check now, not lower the confidence and continue. **Absence of output is not proof of success.** A command that printed nothing needs its exit code shown. +**The output must entail the sentence, not merely agree with it.** Before +writing "verified," read the claim and the pasted output side by side and ask +what the output actually rules out. A run on one version, one host, one image, +one input proves the claim *for that instance*; it does not prove the general +or version-boundary statement the sentence made. When the check comes back +narrower than the claim — and it usually will, because the cheap check is the +reachable instance — rewrite the claim down to what ran, and say the wider one +is still open. Filing a narrow result under a wide heading is the error, even +when every word of the output is true. This binds hardest in a correction: +restating the original overclaim while pasting a narrower proof relabels the +mistake as a fix. Named in logic as hasty generalization, *secundum quid* +(Aristotle, *Sophistical Refutations*, Bk. I ch. 5, trans. W.A. +Pickard-Cambridge, http://classics.mit.edu/Aristotle/sophist_refut.html); in +software it is the difference between a witness and a proof, since one passing +instance witnesses existence and never universality. + **Blaming a gate is a causal claim.** "The hook is wrong," "the check misfired," "the classifier blocked it for no reason" — each one needs the gate's rule read this turn and quoted, with its `file:line`, next to the From 2371e0a52a172e1b19c884d46d608f1ec3dc3d1c Mon Sep 17 00:00:00 2001 From: Edbert Chan Date: Sun, 13 Sep 2026 21:09:34 -0700 Subject: [PATCH 2/2] principle-prove-it: example where the pasted output is narrower than the claim The skill test coverage gate requires a test change with any rule-shaped SKILL.md change. The new entailment rule gets a fires_ example, matching how the subagent-contradiction rule got fires_subagent_contradiction.md. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01HQ9TkZeqARxv3e7W7xesYy Change-Id: Ia60c5c6a87b249bb186b83a5be5b0717603536bd --- .../tests/fires_narrow_output_wide_claim.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 corpus/skills/principle-prove-it/tests/fires_narrow_output_wide_claim.md diff --git a/corpus/skills/principle-prove-it/tests/fires_narrow_output_wide_claim.md b/corpus/skills/principle-prove-it/tests/fires_narrow_output_wide_claim.md new file mode 100644 index 00000000..844c4b67 --- /dev/null +++ b/corpus/skills/principle-prove-it/tests/fires_narrow_output_wide_claim.md @@ -0,0 +1,17 @@ +An agent runs `docker run --rm node:25-slim corepack --version` and pastes the +real output: `corepack: not found`. It then writes "verified: corepack was +removed from Node 25+" into the PR body. One image, one tag, one run. Nothing +checked a second 25.x image, a later version, or the release notes. + +This skill fires. The pasted output is real and every word of it is true, but +it does not entail the sentence. It rules out corepack in that one image; it +says nothing about the version boundary the claim draws. The rule that the +output must entail the sentence, not merely agree with it, is what the reply +needs: rewrite the claim down to what ran ("the `node:25-slim` image has no +corepack") and mark the wider statement as open, or run the check that would +actually cover it. + +The same shape fires on a correction. If the agent later says "I overstated +it earlier, here is the proof" and pastes the same single-image run under the +same "removed from Node 25+" heading, the correction has relabeled the +overclaim as a fix. The narrower proof needs the narrower sentence.