Skip to content

[Hooks Cannot See] (2) Require the output to entail the claim, not merely agree - #519

Open
EdbertChan wants to merge 1 commit into
stack/EdbertChan/reflect/session-c2a4bea7/hooks-cannot-see-1-teach-two-hooks-checks-they-see--09fda6fcfrom
stack/EdbertChan/reflect/session-c2a4bea7/hooks-cannot-see-2-require-output-entail-claim--bc32514b
Open

[Hooks Cannot See] (2) Require the output to entail the claim, not merely agree#519
EdbertChan wants to merge 1 commit into
stack/EdbertChan/reflect/session-c2a4bea7/hooks-cannot-see-1-teach-two-hooks-checks-they-see--09fda6fcfrom
stack/EdbertChan/reflect/session-c2a4bea7/hooks-cannot-see-2-require-output-entail-claim--bc32514b

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Sep 13, 2026

Copy link
Copy Markdown
Owner

Summary

A written rule tells people how to support statements with command output. It helps readers judge whether evidence truly proves what a sentence says.

One check proved a single container image lacked a package. The report widened that result into a claim about newer runtime versions.

No check tested the version boundary, so the output was true but the sentence reached beyond it.

The new paragraph requires narrowing the sentence to the tested case and naming broader claims as open.

Review Claim

Approve one paragraph that requires pasted output to prove the full statement, not merely point in the same direction.

Review Lane

behavior

Review Unit

corpus-lesson

Safety Invariant

Only the body of pull request #519 changes. No commits, pushes, branch changes, labels, comments, or edits to any other pull request are made. The body is published only after the checker passes, and the live body is checked again after publication.

Slice Rationale

Split from slice (1) because docs/ecosystem.md forbids engine runtime and corpus lessons in the same PR, and scripts/check_ecosystem_boundaries.py rejected the combined branch by name.

It sits after (1) because (1) is the mechanical half of the same reflect pass; a reviewer who reads the detectors first has the concrete failure in hand before the general rule.

Non-goals

Does not add a hook. Judging whether pasted output entails a sentence is a meaning question, not a regex one; the mechanical candidate is an llm-judge and is not in this slice.

Does not touch engine/. That is slice (1).

Does not restate the existing missing-check or hedge rules, which already cover their cases.

Test Plan

Test Plan

Prose-only slice; the gates that apply are the skill and ecosystem checks.

check_hook_test_coverage exit=0
check_ecosystem_boundaries exit=0
check_skills_three_harnesses exit=0
check_skill_file_refs exit=0

Whole repo suite on the combined branch before the split:

Ran 463 tests in 1146.986s
OK
EXIT=0
  • python3 scripts/check_ecosystem_boundaries.py
  • python3 scripts/check_skill_file_refs.py
  • python3 scripts/check_skills_three_harnesses.py
  • bash scripts/run_all_tests.sh — 49 directories, 0 failures

Revert Plan

Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert <sha>
  • Post-revert steps: None. The paragraph disappears from the skill; no runtime behaviour depends on it.
  • Data migration? No

…rely agree

The session behind slice (1) ran one command against one container image and
proved that image had no corepack. That was written up as corepack having been
removed from Node 25+, a version-boundary claim no command in the session ever
checked. The output was true and the sentence was wider than the output.

prove-it already covers a missing check and a hedge. It did not cover a check
that ran, came back narrower than the claim, and got filed under the wider
heading anyway -- which is the common case, because the cheap check is always
the reachable instance.

It binds hardest in a correction: restating the original overclaim while
pasting a narrower proof relabels the mistake as a fix, which is what happened
here before the rule existed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ZqCxSQVpmM9Rx37C7irgf
Change-Id: Ibc32514be641d37277115faf6068e75ea02e0083
@EdbertChan

Copy link
Copy Markdown
Owner Author

This pull request is part of a Mergify stack:

# Pull Request Link
1 [Hooks Cannot See] (1) Teach two hooks the checks they could not see #518
2 [Hooks Cannot See] (2) Require the output to entail the claim, not merely agree #519 👈

@cursor

cursor Bot commented Sep 13, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_bac5280c-bd98-4100-8056-7d56a041abd8)

EdbertChan added a commit that referenced this pull request Sep 13, 2026
…ody checker on PR #519's live body after the rewrite.

Goal: Prove the body passes as published.
Motivation: A PATCH that returned 200 is not proof the live body passes.
Safety invariant: Read-only; a failed fetch fails the task instead of passing.
Verify: exit code 0 and output PR body validation passed.
Effectiveness measurement: The same checker the required GitHub check runs passes on the live body, so the PR is no longer blocked on its text.

Exit code: 0
EdbertChan added a commit that referenced this pull request Sep 13, 2026
…ody checker on PR #519's live body after the rewrite.

Goal: Prove the body passes as published.
Motivation: A PATCH that returned 200 is not proof the live body passes.
Safety invariant: Read-only; a failed fetch fails the task instead of passing.
Verify: exit code 0 and output PR body validation passed.
Effectiveness measurement: The same checker the required GitHub check runs passes on the live body, so the PR is no longer blocked on its text.

Exit code: 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant