Skip to content

chore(attribution): restore the bundled fixture exclusions to close the provenance sweep - #13643

Merged
kyle-sexton merged 3 commits into
mainfrom
chore/3465-provenance-sweep-close
Nov 6, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
chore/3465-provenance-sweep-close

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Refs: #3465

Summary

This PR delivers the fixture disposition for the provenance sweep sub-topic: the repository config now lets the attribution plugin's bundled test-fixture exclusion apply. The sweep that produced it, attribution-3465-close-20261106T023054Z, also covered the two other gaps in #3465's 2026-11-06 close audit. That evidence is in an issue comment (record), not in this diff.

Each #3465 checklist item and where it stands:

Item State Evidence
Golden set past min_n_per_class met by owner ruling near-verbatim n=15; the owner's 2026-11-02 ruling makes near-verbatim fix-eligible on its own (close audit, 2026-11-06)
Report-only sweep done the 2026-10-28 and 2026-11-04 sweep records on #3465
Live no-breadcrumb not-found probe done budgeted search on check-catalog.md:414-419: search 1 of 3 found the source, fetch 1 of 5 confirmed it (record)
Sweep resume semantics done an interrupted and resumed sweep restored spend 3, refused a re-close (exit 3), and re-validated the cache (same record)
Disposition-applying sweep done ran 2026-11-04 (applied 0); this sweep closed the fixture leave-with-reason, and the durable keep is this PR
Convention engagement, hash-store trigger done #13461
Dead-pointer round-trip done #13549 with melodic-software/ci-workflows#993
Narrow CI regression gate deferred owner decision Q-9759-2 (b), recorded by #13595

Fix

  • docs/conventions/attribution.yaml: excluded_paths is removed. A layer's list replaces the bundled default whole (plugins/attribution/schemas/attribution.schema.json:12), so the repository's single **/evals/fixtures/** entry had silently dropped that default. The default also behaves differently from a copied list: it is matched below the scan target and skipped for a file named directly (list-corpus.sh:15-16, :305-313). The first cut copied its globs instead, and review showed that would decline a named fixture.
  • The config comment says the eval harness's config isolation lifts the default, so eval fixtures still report their real findings (list-corpus.sh:17-19).

Verification

  • list-corpus.sh on this branch: a repo-wide run declines 485 files under the bundled default reason, up from 277, so 208 more files in fixture and testdata trees are no longer audited as prose. scripts/fixtures/hooks-lifecycle-table.md named directly reads 1 included, 0 declined.
  • The fixture itself is unchanged, and no per-file suppression was added.
  • Left out: no eval run, because no skill or script changed.

Remaining

Related

Follow-ups

  • Pre-existing (advisory): plugins/machine-health/skills/audit/reference/windows/check-catalog.md:414-419 restates the Task Scheduler constants. The recommended disposition is condense-to-stamped-record, pointing at Microsoft's constants page, for a human to apply.
  • Pre-existing (advisory): the attribution skill finds its config and sweep ledger from the session's working directory, so a run aimed at another worktree by path reads the wrong checkout's config.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RMa441wb9nbFwyTS8nqdYj

…epository's config

docs/conventions/attribution.yaml set excluded_paths to "**/evals/fixtures/**" before the
plugin's 0.16.0 default declined every test-fixture tree. A layer that sets the key replaces the
default whole (plugins/attribution/skills/setup/reference/keys.md "The test-fixture
exclusion"), so script fixtures such as scripts/fixtures/hooks-lifecycle-table.md, a saved copy
of the Hooks reference lifecycle table that scripts/gen-hook-event-registry.test.sh reads, were
audited as prose and reached every fix gate.

The list now carries the default's three globs. The fixture's disposition is a categorical
keep, not a per-instance suppression. The repository corpus goes from 277 declined files to
485: 462 under **/fixtures/**, 2 under **/testdata/**, 21 vendored.

Refs: #3465

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RMa441wb9nbFwyTS8nqdYj
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 6, 2026 02:37
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-11-06T02:37:48.597751Z
We couldn’t complete this review. Comment @codex review to try again.
000a7b7 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed cb2ef0811d343a4767237d285e5ed2852e4adb25: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental, blocking only): no security issues found

Scope: I read all of the change since b7e4bfb7 to the one listed file, docs/conventions/attribution.yaml. It matches none of the trusted-path globs. This lane already reviewed an earlier head (blocking only: yes), so this review reports only CRITICAL and IMPORTANT findings.

What changed: only the YAML header comment. It now says that the eval harness's config isolation lifts the bundled test-fixture default, so eval fixtures still report their real findings, and it cites the list-corpus.sh header. No config value changed: excluded_paths is still unset and trigger_less_stamp_check: true is unchanged.

What I checked:

  • Behaviour: none changed. The change is comment text in a config file that only the attribution plugin's audit scripts, tests and evals read. No workflow, merge hold, kill switch, trusted-actor check or token broker reads it.
  • Security impact: none. No secret is exposed, no token's scope grows, no new actor is trusted, no untrusted text reaches a lane that holds credentials, and nothing can bill the owner.
  • Instruction surfaces: no standing instruction is deleted or weakened.
  • Plugin manifests: no plugin.json or marketplace.json changed.
  • PR context and settled threads: neither contains embedded instructions. The settled thread on this file (copied fixture globs) was fixed at b7e4bfb7, and this push doesn't reopen it.
  • Prior findings: the earlier reviews by this lane found nothing, so nothing carries forward.

Spend: none.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 000a7b7523a5fe2f455970079eeea8601169c6fd; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread docs/conventions/attribution.yaml Resolved
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed cb2ef0811d343a4767237d285e5ed2852e4adb25: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #13643, changes since b7e4bfb7 (blocking only)

Result: no findings. This lane had already reviewed an earlier head, so this pass looked for blocking findings only.

What I read: the one file in scope, docs/conventions/attribution.yaml. I read every hunk of its change.

What changed: only the header comment. It gains one sentence saying that the eval harness's config isolation lifts the bundled test-fixture default, so eval fixtures still report their real findings. The sentence cites the header of list-corpus.sh. No YAML keys or values changed: excluded_paths is still unset and trigger_less_stamp_check: true is unchanged.

Is the comment accurate? Yes. The header it cites says so in two places:

  • list-corpus.sh#L17-L19: "the harness lifts the default either way and the fixtures report real findings".
  • L21-L27: the harness reads its config from a separate root, which is the isolation the new sentence refers to.

Earlier threads: the lane's earlier excluded_paths thread stays resolved. This push does not reopen it, because it changes no config.

Closing issue #3465: nothing changed since the last review. The close evidence is the author's record of sweep runs, which I did not verify.

Not applicable: no plugin manifest changed. I found no embedded instructions in the PR context, the settled threads or the prior findings.

… applies

Review on #13643: copying the default's globs into this layer changed
their behavior. list-corpus.sh matches a config-layer list against full
repository paths, so a named fixture is declined. The bundled default is
matched below the scan target and skips a named file. With the key unset,
fixture and testdata trees are still declined in a repo-wide run, and a
run aimed at a fixture reads it.

Refs: #3465

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RMa441wb9nbFwyTS8nqdYj
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

The pr-verify verifier failed head b7e4bfb731b4ce457b3a45283d7fed5f8b41ed92. What it wrote, as data:

What it wrote (28 lines)
Summary: The PR says `Closes #3465`, but its diff only removes `excluded_paths` from
  `docs/conventions/attribution.yaml`, so the bundled test-fixture default exclusion applies again.
  As a config change it looks sound. It is categorical, adds no per-instance suppression, and
  touches no secret, setting or takeover file. Every changed path is covered by a green job:
  lint-repo, check-plugins and test-bash (0) all passed
  (https://github.com/melodic-software/claude-code-plugins/actions/runs/42152259071/job/127679317535).

The problem is scope. #3465 is the sub-topic Brief, and its spec entry has no `person_checks`, no
  `credited_siblings` and no `scope`, so every checklist item counts in full. Several items are not
  delivered by this diff and are not excused for it. The issue's own 2026-11-06 close audit says the
  narrow CI regression gate for the cleaned state was moved to #9759, so nothing has delivered it.
  The convention engagement and hash-store trigger (#13461) and the dead-pointer wiring (#13549)
  shipped in other PRs, but the PR body has no `Siblings:` line, so they are not credited.
  Golden-set growth past `min_n_per_class` for verbatim (n=5) rests on an owner ruling, not on this
  change. The sweep-run items (budgeted search, interrupted-and-resumed sweep) are recorded in an
  issue comment, not in the diff. Closing the issue here would close it while checklist work is
  still open; the issue was already closed in error once before (#4960). Neither the PR body nor the
  diff contains an evidence-loop cap line or any text trying to steer the reviewer.
- PR body: change `Closes #3465` to `Refs: #3465`. Say that this PR delivers the fixture disposition
  (bundled fixture exclusion restored), and name what remains, at least the cleaned-state CI gate in
  #9759. If you keep `Closes`, add a `Siblings:` line naming the merged PRs that delivered the other
  checklist items (#13461, #13549), and make sure every remaining item, including the CI regression
  gate, is actually delivered first.
- docs/conventions/attribution.yaml:5: the deleted comment said the fixtures exclusion lived in
  config so the eval harness's config isolation would lift it. Confirm that the bundled default does
  not now hide `**/evals/fixtures/**` inputs during attribution eval runs (or that the
  scan-target-relative matching keeps them visible). Then state that in the new comment, so eval
  fixtures still report their real findings.

This pull request leaves the merge queue, and ci-status holds it out while it carries pr-verify-failed. A pr-verify pass removes the label, at a new commit or at this same head: once a trusted person edits the body, and only trusted people have edited it since the fail, this head is judged again, so a fail on the body or the issues it links needs only the body fixed, no new commit.

@github-actions github-actions Bot added the pr-verify-failed pr-verify failed this head; a pass at a new commit, or after a trusted body edit, removes it. label Nov 6, 2026
pr-verify on #13643: the earlier comment said the eval harness's config
isolation lifts the fixture exclusion. The bundled default keeps that
property (list-corpus.sh header), and the config comment says so again.

Refs: #3465

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RMa441wb9nbFwyTS8nqdYj
@kyle-sexton
kyle-sexton removed this pull request from the merge queue due to a manual request Nov 6, 2026
@github-actions github-actions Bot removed the pr-verify-failed pr-verify failed this head; a pass at a new commit, or after a trusted body edit, removes it. label Nov 6, 2026
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 6, 2026
Merged via the queue into main with commit a7d7e5a Nov 6, 2026
53 checks passed
@kyle-sexton
kyle-sexton deleted the chore/3465-provenance-sweep-close branch November 6, 2026 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant