[kyle-sexton]

Refs: #13586

Summary

post-merge-test-mutation.yml moves off claude-code-action, the last lane in #13586 without the three-leg chain. Before, the model ran every mutant in its own Bash with the automations OAuth token in the job. Now the owner's constraint holds: no job that runs mutants holds a Claude credential or a write token. Two credentialed run-claude-cli chains, generate and triage, sit around a mutant job that holds no secret, and they pass data through artifacts pinned by SHA-256. The lane is still advisory, gates nothing, and a usage or capacity limit is still a designed stop.

Job	Before	After
audit	one claude-code-action session with the subscription token; generates, runs and triages mutants	removed
generate, generate-subscription, generate-fallback	none	key, then subscription, then fallback; --phase generate; read-only, runs no test
mutate	none	no secret, contents: read; applies and runs each mutant
triage, triage-subscription, triage-fallback	none	the same chain; --phase triage over the survivors; starts only when a mutant survived
key-credit	none	check-key-credit for both chains, actions: read only
report	reads the audit's structured output	assembles the result from the mutate job's states and triage's verdicts, then writes the same report

Fix

The skill's split.

• The gap. /mutation-testing:audit had no way to stop after Phase 2 or to start at Phase 4.
• The flag. It gains --phase generate|triage, with the detail in skills/audit/context/split-runs.md and a minor changelog fragment:
  • generate: Phases 0 to 2 without running any test. It returns each mutant's path, line, operator, old_string and new_string.
  • triage <file>: Phase 4 over the survivors the executor ran. It returns verdicts by index.

The mutant job. mutate runs .github/scripts/post-merge-test-mutation/execute.mjs.

• Input. It checks the generation against generate.schema.json.
• Validation. It applies a mutant only when it names a tracked file inside the config's mutate globs and its old_string occurs exactly once and covers its line. Any other mutant is invalid and is never applied.
• Baseline and runs.
  • Each file gets one covering-tests baseline. A red or timed-out baseline leaves that file's mutants not-run, and exit 2 means no-coverage.
  • Each mutant then runs once under the larger of timeout-ms and 1.5 times the baseline plus 5 s. The harness exit gives the state: 0 is survived, 1 killed, 2 no-coverage, a timeout timeout.
  • The harness runs from a read-only copy of scripts/, as before.
• Restore. After every mutant the original bytes go back, and the tracked tree must match its starting state. The first mismatch ends the run restore-failed. The job's own git diff --quiet HEAD check stays.
• Holds. The job holds no secret and no write or OIDC grant, and every checkout is persist-credentials: false. Its scanner comes from a fresh checkout of main after the mutants run.

Report.

• assemble.mjs takes every state from the executor and only a survivor's disposition and cause from triage. A survivor triage gave no verdict is unclassified.
• Both inputs are checked against their schemas, and report.mjs is unchanged.
• A limit stop on the last leg that runs, of either chain, reports as the designed stop. Any other failed leg fails the run.

Chains. They follow the pattern of #13590, #13592 and #13608:

• one credential per job;
• an empty subscription token hands to the fallback leg, which starts only when its token is set;
• every leg writes a report-lane-outcome record with lane-id;
• each key leg uploads key-credit-stop-post-merge-test-mutation.yml-<generate|triage> after a credit stop;
• every credentialed job's if: keeps the same-repository workflow_run gate as a top-level term;
• every leg uploads its sealed transcript, so the audit job's TRANSCRIPT_EXEMPT entry is gone.

Catalog and tests.

• Catalog. lane-funding.json lists the six legs, and .github/artifact-retention.json has rows for the generate, executed and triage artifacts and the two markers.

• Exemption. The post-merge-test-mutation.yml automations audit LEG_EXEMPTIONS entry is gone. With ci(lanes): run the maintenance eval and propose lanes on the three-leg chain #13626 merged, LEG_EXEMPTIONS is now {}.

• The no-secret test (callers.test.mjs). It fails when any job that runs mutants (a step running run-covering-tests.sh or execute.mjs):

  • reads any secret, in any form secretRefs sees;
  • names github.token or GITHUB_TOKEN;
  • grants write or id-token, or inherits the workflow's permissions;
  • keeps a checkout credential.

  Its fixture shows each case failing, and it asserts that the mutation lane's mutate job is found.

• Other tests.

  • a leg-shape test holds each chain's three legs equal;
  • execute.test.mjs runs the executor against a real temp git repository with a stub harness;
  • report.test.mjs's workflow tests follow the new jobs.

Verification

• Node suites. node --test ".github/scripts/**/*.test.mjs" ".github/actions/**/*.test.mjs" (this covers .github/scripts/lib): 8221 tests, 8221 pass, 0 fail, 0 skipped after merging main with ci(lanes): run the maintenance eval and propose lanes on the three-leg chain #13626. The post-merge-test-mutation tests run 48 of 48.

• Bash suites:

  Suite	Result
  write-runner-never-rules.test.sh	76/76
  check-artifact-retention.test.sh	39/39
  check-app-key-references.test.sh	9/9
  check-read-caller-keys.test.sh	68/68
  check-lane-coverage.test.sh	41/41
  check-scan-carrier-secret-shadow.test.sh	9/9
  workflow-self-paths.test.sh	132/132
  credential-sweep.test.sh	75/75
  run-zizmor.test.sh	13/13
  ci-review-lanes.test.sh	245/245
  plan-test-lanes.test.sh	58/58
  run-covering-tests.test.sh	7/7
  affected-tests-live.sh	39/39

• Gates, all clean:

  • runner-policy.mjs --root . (passed, so no standards contract change is needed);
  • check-artifact-retention.sh --check (129 uploads, 87 families), check-read-caller-keys.sh and check-app-key-references.sh;
  • actionlint and zizmor;
  • the changelog fragment checks, validate-plugins.sh --only mutation-testing and check-changed-skills.sh;
  • the whole-repo scanners: silent-skips, discriminating-test-skips, test-tmp-cleanup, lane-coverage, cant-fail and docs-only.

• Simulated report step. The rendered report step ran in bash with the real assemble.mjs and report.mjs:

  Input	Result
  survivor with a triage verdict	reported, exit 0
  survivor with no triage	fails
  executed file whose SHA-256 differs	fails
  limit stop	designed stop, exit 0

Remaining

This PR delivers the post-merge-test-mutation lane. Already merged for #13586: #13590, #13592, #13594, #13608, #13610, #13626 and #13636. What remains on #13586 is intake triage's check-key-credit gate, melodic-software/ci-workflows#996. It waits on melodic-software/standards#950, which accepts both caller permission sets, then the standards sync and the intake caller's actions: read grant.

• UNVERIFIED on a hosted runner:
  • the skill following --phase generate and --phase triage under run-claude-cli --restricted, with the plugin loaded from .lane/plugins through plugin-dirs;
  • Phase 4's subagent through the Agent tool in the triage legs.
• Residual exposure. The mutate job's head code can rewrite files later steps in that job read, the action code in _work/_actions included. It holds no secret, no write grant and cache-mode: read, and downstream jobs treat its output as untrusted, schema-checked data. So the worst a mutant can do is a wrong advisory report.

Related

• ci(lanes): run every model lane on the same key, subscription and fallback chain #13586
• ci(lanes): run the maintenance eval and propose lanes on the three-leg chain #13626 (the maintenance eval and propose lanes)
• ci(lanes): run the release, adjudicate, refine and capture lanes on the three-leg chain #13590, ci(lanes): run the code-tidying upkeep sweep on the three-leg chain #13592, ci(lanes): give every three-leg lane the key-credit gate and the empty-subscription handoff #13608, test(lanes): fail a model lane that lacks its key, subscription or fallback leg #13594

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-06T00:44:02.645463Z	d6fd0c6	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6fd0c66e0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

claude-security-review reviewed d9895b425b0e8f1127d39f494d7ba888f83fd8ba: read 6 of 6 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #13632, incremental (d6fd0c6 → d9895b4)

No security issues found in this push.

I read every hunk of all 6 files on the incremental list. .github/workflows/post-merge-test-mutation.yml is a trusted path, so I read its whole change and the generate-leg steps around it. This lane already reviewed an earlier head (blocking only: yes), so only CRITICAL and IMPORTANT findings would be posted. There were none of any severity.

critical-findings: 0

What I checked

Generate legs can now run the suppression lint. The new rule is Bash(bash ${{ steps.prompt.outputs.lint }} *) at L394, L722 and L1008.

• The rule cannot collapse to Bash(bash *). That would happen if the lint output were empty. The run step is gated on steps.prompt.outcome == 'success', and the same Stage the plugin and the prompt step always writes lint= before it succeeds (L348).
• The script is main's own code and cannot be altered. It is copied from .base (main), checked for symbolic links and set read-only with chmod -R a-w (L334-L337).
• The script writes nothing. suppression-lint.sh only reads and parses the record paths it is given. Its one file write is a mktemp buffer for stdin, removed on exit.
• It is not a way to read secrets. It prints a file's content only after a top-level suppressions: mapping, and only as parse output. So it does not expose files such as /proc/self/environ, whose Read is denied.
• It matches what was already reviewed. The triage legs already held this exact rule at the earlier head. Every other disallowed-tools deny (Edit, Write, NotebookEdit, WebFetch, WebSearch, Agent, mcp__*, the git-read limits) is unchanged. report.test.mjs now requires the lint rule on every model leg and still allows only the read-only git verbs (status, diff, branch, rev-parse).
• Only main's merged code reaches these legs. They run with contents: read, and the run starts only from a push to main. No outside actor reaches them.

execute.mjs glob matcher. path.matchesGlob is replaced by the new globPattern, whose ** also matches directories that start with a dot.

• Special characters are escaped. Everything other than *, **/ and ? is escaped before it goes into the regex, [ and ] included.
• The pattern is anchored at both ends.
• The excluded trees stay out. Negated globs still win when they come later. The new test checks the repository's own config and confirms .claude/**, docs/conventions/**, **/evals/**, **/tests/** and *.test.* stay out of scope.
• The wider scope runs nowhere sensitive. It admits sources under .github/ to mutation. Mutants run only in the mutate job, which reads no secrets and has contents: read. The other checks on mutant files (tracked file only, no . or .. segments, schema, a unique old_string) are unchanged.

Instruction surface. generate.prompt.md now says Bash may run the suppression lint, matching the new grant. The argument-hint change in audit/SKILL.md only edits the hint: --phase is dropped from it and --full added. Neither change removes a guardrail. I did not consult the protected-class register; recognition by consequence found nothing lost.

Trust controls. None changed: trusted-trigger check, lane token request and broker, kill switch, merge holds and resolve-config.

Prior findings. The only earlier post is the summary comment at d6fd0c6, which reported no findings. There is nothing to repeat, and the settled threads hold nothing this push touches.

[github-actions]

claude-review reviewed d9895b425b0e8f1127d39f494d7ba888f83fd8ba: read 6 of 6 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #13632, incremental review (d6fd0c6 → d9895b4)

This lane already reviewed an earlier head (blocking only: yes), so this review reports blocking findings only. I read every hunk of all 6 files in .claude-lane/incremental.diff. For context I also read docs/conventions/mutation-testing.yaml, the prompt steps of post-merge-test-mutation.yml that set outputs.lint, and the --phase entry in the audit SKILL.md.

No blocking findings.

What changed, and how it holds up:

• The generate legs can now run the suppression lint. This resolves the earlier advisory finding chore: initialize marketplace scaffold #1. Each of the three generate legs (post-merge-test-mutation.yml lines 394, 722 and 1008) now allows Bash(bash ${{ steps.prompt.outputs.lint }} *). The prompt step of each job sets lint to the plugin's suppression-lint.sh (lines 348, 675 and 962), so the rule never collapses to a bare bash *. The test in report.test.mjs now requires that one rule on every model leg, and the comments and generate.prompt.md match.
• globPattern replaces path.matchesGlob in inScope. I traced every glob in the repository's mutate list (**/*.ext, !**/*.test.*, !**/test_*.py, !**/conftest.py, !**/x/**, !.claude/**, !docs/conventions/**) through the new matcher. Each one gives the intended anchored match, with dot directories included. The matcher has no support for braces or character classes (it escapes them as literals), but no glob in the config uses them. The new test reads the real config, so it fails if the scope changes. The owner reply in the settled thread on execute.mjs covers this change.
• --full is back in the audit SKILL.md argument-hint. This resolves the earlier advisory finding fix: correct plugin-system inaccuracies and harden references #2. The same edit drops --phase from the hint, though SKILL.md line 64 still documents that flag. I am not raising it, because this review is blocking only and it does not block.

Closing issue #13586: this push does not change how the earlier review judged the criteria.

Not checked by me: I ran nothing. Test results and hosted-runner behaviour are the author's claims.

Spend (not a hold): none new in this push. The extra lint calls in the generate legs use only the existing prepaid credit or subscription usage.

[github-actions]

The pr-verify verifier failed head d9895b425b0e8f1127d39f494d7ba888f83fd8ba. What it wrote, as data:

What it wrote (44 lines)

Summary: Issue #13586 asks for nine lanes to be brought onto one three-leg chain: role API key, then
  role subscription, then FALLBACK_CLAUDE_CODE_OAUTH_TOKEN. It also asks for a test that enforces
  the chain on every lane, for the release lane's credit-stop skip and header to be fixed, and, in
  later comments, for the check-key-credit gate on every lane and a handoff classifier for the
  `claude plugin eval` lanes. This PR says `Closes #13586`, but the diff only changes
  post-merge-test-mutation.yml.

For that one lane, the work looks sound:
- **Legs.** The lane moves from claude-code-action to run-claude-cli with two chains, generate and
  triage. Each chain has key, subscription and fallback legs, holds one credential per job, keeps
  the same-repository gate and uses the check-key-credit gate.
- **Mutant job.** The mutants run in a job that holds no secret, has `contents: read` only and keeps
  no checkout credential.
- **Handoff data.** Data passes between jobs as artifacts pinned by SHA-256 and is checked against
  schemas.
- **Tests and exemptions.** The audit job's `LEG_EXEMPTIONS` entry and the `TRANSCRIPT_EXEMPT` entry
  are removed, which tightens the existing checks. New tests cover the no-secret rule for mutant
  jobs and the shape of each chain's legs, and execute.test.mjs runs against a real temporary git
  repository.
- **CI.** `test-node` and all `test-bash` jobs passed on the judged head
  (https://github.com/melodic-software/claude-code-plugins/actions/runs/42140699743/job/127640158422).

I found no correctness or safety defect in the changed code. The change to the resolve-config
  callers test only makes it stricter. I found no text that tries to steer a reviewer.

The verdict is fail because nothing in spec-text.json excuses the other parts of the issue: no
  `person_checks` and no `credited_siblings`. This PR's body lists #13590, #13592, #13608, #13594
  and #13626 only under "Related", not on a `Siblings:` line, so those PRs can't be counted as
  delivering the other parts. Without that, these done-when items are not delivered by this diff:
  the release lane's (upstream-apply-changelog.yml) credit-stop skip and header, the chain for the
  other eight lanes, the key-credit gate on lanes that already had all three legs, and the handoff
  classifier for the eval lanes.
- PR body: it says `Closes #13586` but the diff only covers post-merge-test-mutation.yml. Add a
  `Siblings:` line naming the merged PRs that carry a `Refs: #13586` line and delivered the other
  lanes (for example #13590, #13592, #13608, #13626, #13594), so their parts can be credited. If
  some part is still undelivered, change the line to `Refs: #13586` and name the remaining work.
- .github/workflows/upstream-apply-changelog.yml: the done-when item 'the release lane's credit-stop
  skip is removed or reduced to the shared key-credit gate, and its header no longer says it has no
  subscription leg' is not in this diff, and no credited sibling covers it. Credit the sibling PR
  that delivered it through a `Siblings:` line, or deliver it here.
- Issue #13586 comments: the scope additions (the check-key-credit gate on lanes that already had
  all three legs, and a handoff classifier for the fallback leg of the `claude plugin eval` lanes)
  are not in this diff and no credited sibling covers them. Credit the sibling PRs that delivered
  them through a `Siblings:` line, or deliver them here.

This pull request leaves the merge queue, and ci-status holds it out while it carries pr-verify-failed. A pr-verify pass removes the label, at a new commit or at this same head: once a trusted person edits the body, and only trusted people have edited it since the fail, this head is judged again, so a fail on the body or the issues it links needs only the body fixed, no new commit.