Skip to content

ci(test-lanes): run every package whose suites read every workflow on a workflow change - #13925

Merged
kyle-sexton merged 3 commits into
mainfrom
fix/select-tests-workflow-readers
Nov 7, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
fix/select-tests-workflow-readers

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: ledger L377 (test selection misses suites that read every workflow), left from #13642, #13645 and #11154

Summary

scripts/plan-test-lanes.sh selected a test package on a workflow change only when one of its suites named that workflow file. Six packages have suites that list .github/workflows/ and check every workflow, so a change to a workflow no suite names skipped them. For example, -- .github/workflows/maintenance-delete-stale-branches.yml selected only branch-cleanup and lane-transcripts on main.

Fix

  • .github/workflows/ is added to resolve-config's SUBPROJECTS line, and OUTSIDE_TRIGGERS rows are added for check-signed-commits, check-wip-cap, log-api-budget, .github/scripts/lib and pr-merge. This is the selector's existing mechanism for the same gap (ci(test-lanes): run lane-transcripts on any workflow change #13658).
  • A generic scan was not used: a suite that lists the directory names no workflow, and the listing calls vary (a readdirSync grep missed pull.test.mjs).

Verification

  • scripts/plan-test-lanes.test.sh: 60 of 60 pass. Against the old planner the extended suite gives 58 pass and 2 fail: "a workflow no suite names runs every package whose suites read every workflow", and the pr-verify.yml case.
  • Probe after the fix: -- .github/workflows/maintenance-delete-stale-branches.yml selects all eight packages.
  • selection-audit.test.sh 20 of 20; the three Node suites that mention the planner 256 of 256; shellcheck clean.

Related

Follow-ups

  • Pre-existing (advisory): the fix is per package, so a new suite that lists the workflows directory will be missed again, the second time after ci(test-lanes): run lane-transcripts on any workflow change #13658. A // test-scope: declaration for Node suites (affected-tests.sh R8 accepts only # comments today) would close it for good.
  • Pre-existing (advisory): every workflow change now also runs these six packages, including resolve-config's large callers.test.mjs; the added wall time is not measured.

session: automation-lanes

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits November 7, 2026 02:12
… a workflow change

A Node suite that lists .github/workflows/ with readdirSync names no
workflow, so the selector never reaches it from one: a change to an
unnamed workflow (maintenance-check-upstream-drift.yml,
maintenance-delete-stale-branches.yml) skipped resolve-config's
callers.test.mjs and api-budget-coverage.test.mjs, which assert across
every workflow. #13658 fixed the same gap for lane-transcripts alone.

Give each such package the .github/workflows/ trigger: resolve-config,
check-signed-commits, check-wip-cap, log-api-budget, .github/scripts/lib
and pr-merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 7, 2026 07:18
@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-07T07:18:32.018280Z
We couldn’t complete this review. Comment @codex review to try again.
5923b3a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 6a390d5d1aea455c264a69725139676b148c5d3c: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #13925 (incremental, blocking only)

Scope: This is an incremental review covering changes since 5923b3ad. One file changed, scripts/plan-test-lanes.test.sh, and I read its whole change (one hunk). It does not match any trusted-path glob. This lane has reviewed an earlier head of this PR, so this round reports only CRITICAL and IMPORTANT findings.

What changed: The test case for a change to .github/actions/check-kill-switch/action.yml gained one extra assertion: is "$(key node)" false. It checks that a change outside .github/workflows/ selects no Node packages at all, in addition to the existing check that it selects none of the packages that read every workflow.

Security assessment: I found no security issues. The change only adds an assertion to a test of the CI test-lane planner. It does not touch:

  • the trusted-actor list, the token request, the kill switch or any merge hold
  • any manifest
  • any standing instruction

The kill-switch path in the hunk is only a test fixture input; the kill switch action itself is unchanged.

Prior findings: The earlier summary comment reported 0 findings, so nothing carries over.

The PR context contains no embedded instructions aimed at this lane.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 5923b3ad0b7ce7cae3106a2e7fb49f6e6c1e3546; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread scripts/plan-test-lanes.test.sh Resolved
@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 6a390d5d1aea455c264a69725139676b148c5d3c: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #13925 (incremental since 5923b3ad, blocking only)

Scope: This was an incremental review. I read the only hunk of the only file in scope, scripts/plan-test-lanes.test.sh. This lane already reviewed an earlier head, so only blocking findings would be reported.

What changed: In the "a change outside .github/workflows/" case, the test now also asserts is "$(key node)" false. That is the suggestion from this lane's earlier advisory finding (discussion_r4500640059), applied as written.

Check: The fix closes that gap.

  • The planner prints node=false only when it completes with an empty node_packages (plan-test-lanes.sh:447).
  • If the planner crashes, OUT is empty, so key node returns "", is "" false fails, and the check reports the failure.

No new findings.

Verification: I ran nothing (this lane has no shell). The PR's test counts are the author's own figures and I have not checked them.

Closing issue: none. The PR cites ledger L377, so there are no acceptance criteria to check.

Lane-Activity: pr-unblock/address-feedback
Lane-Thread: PRRT_kwDOTCGFQM60zbEz
@melodic-automation-lanes

Copy link
Copy Markdown
Contributor

pr-unblock / address-feedback finished round 1 of max 4 on 6a390d5.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 7, 2026
Merged via the queue into main with commit d70448f Nov 7, 2026
55 checks passed
@kyle-sexton
kyle-sexton deleted the fix/select-tests-workflow-readers branch November 7, 2026 11:27
@github-actions github-actions Bot added the merge-path:owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path:owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant