Skip to content

docs(contributing): do not cancel the CI run of the PR being merged (4 PRs through this hole in one day) - #9256

Merged
proggeramlug merged 2 commits into
PerryTS:mainfrom
proggeramlug:docs/dont-cancel-the-merging-pr
Aug 31, 2026
Merged

docs(contributing): do not cancel the CI run of the PR being merged (4 PRs through this hole in one day)#9256
proggeramlug merged 2 commits into
PerryTS:mainfrom
proggeramlug:docs/dont-cancel-the-merging-pr

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Docs only, one paragraph in CONTRIBUTING.md. Raised by @perry-99 during the v0.5.1519 freeze; filed as a PR so Ralph can rule on the policy.

Four PRs through one hole in a single day, confirmed from both ends

Broken code in through one end, release-note fragments silently dropped out through the other — one mechanism, observed twice, on the same day. One occurrence is an accident; four is a hole.

Those three are mine, which is how the second failure mode surfaced: I went looking for why my own fragments were missing and found that the check enforcing them never ran. That makes it a process defect rather than a contributor error — the convention cannot be violated knowingly if nothing reports the violation.

The rule

Cancelling other runs for capacity is fine and expected on a 20-slot org. Cancelling the run of the PR you are about to merge is not, because a cancelled job is neither a pass nor a failure, and two protections go quiet at once:

  1. pr-gate never reports — the single required context is absent rather than red, which is what invites the admin bypass, and the bypass is what lands the change.
  2. The changelog fragment stops being required — that check is a step inside lint conditioned on github.event_name == 'pull_request', so a cancelled lint skips it silently, and the omission stays invisible until the notes are cut.

The check to teach: pr-gate present and passing

Not "nothing is red". Those are different assertions and only the first is worth anything:

  • A gate that never ran is absent from the status list, and CANCELLED is neither pass nor fail. Both read as clean under any check that looks for failures.
  • pr-gate: pass is a positive statement that the fan-in ran and every dependency was success or skipped. "0 failing" is satisfied equally well by a PR whose gate never executed.

I made exactly that mistake, twice over: I reported one of these PRs as "29 checks, failing: none" while its lint sat cancelled. @perry-99 audited their release tooling against this and has made the dispatch condition require full-suite-gate conclusion success plus zero cancelled jobs — the same hole exists one level up, because a skipped or absent required context also satisfies "not failing".

What this deliberately does not propose

No change to the gate. gate in .github/workflows/test.yml (line ~3746) already runs if: always(), and its documented verdict is "every other needed job must be success or skipped; failure and cancelled fail" — precisely so a cancelled dependency cannot read as green. Every incident here is a bypass of a gate that works. The text says so explicitly, so nobody reads this and goes hunting for a gate bug.

Refs #9169, #9247, #9215, #9230, #9235, #9255.

Summary by CodeRabbit

  • Documentation
    • Added contributor guidance to avoid cancelling CI runs for pull requests being merged.
    • Documented how to interpret gate results and verify successful release checks.
    • Added notes on incidents where cancelled checks affected release validation.

A cancelled job is neither a pass nor a failure, and two protections go
quiet together: pr-gate never reports (so the required context is absent
rather than red, which is what invites the bypass), and the changelog
fragment check — a step inside lint, conditioned on pull_request — is
skipped silently, so the omission stays invisible until release notes are
cut.

Both were observed on the same day. PerryTS#9169 merged with lint failing and five
jobs cancelled, breaking method dispatch and property lookup on main for
four and a half hours (PerryTS#9247). PerryTS#9215, PerryTS#9230 and PerryTS#9235 each merged with lint
CANCELLED; all three touched crates/, none carried a fragment, and the work
is absent from its release notes.

States explicitly that the gate is correct and should not be changed: gate
in test.yml runs if: always() and treats cancelled as failure, exactly so a
cancelled dependency cannot read as green. Every incident has been a bypass
of a working gate.

Docs only.
@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4b90186a-7f70-46a2-9ebc-ce7ef5263f97

📥 Commits

Reviewing files that changed from the base of the PR and between 8375481 and 17beb3d.

📒 Files selected for processing (1)
  • CONTRIBUTING.md

📝 Walkthrough

Walkthrough

Changes

CI Cancellation Guidance

Layer / File(s) Summary
Document CI cancellation effects
CONTRIBUTING.md
Adds merge-safety guidance for cancelled PR runs. It documents pr-gate, changelog-fragment validation, affected incidents, and the requirement to verify a successful gate conclusion.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Suggested reviewers: thehypnoo

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

… red'

A gate that never ran is absent from the status list, so it reads as clean
under any failure filter — the same way CANCELLED does. 'pr-gate: pass' is a
positive assertion that the fan-in ran and every dependency was success or
skipped; '0 failing' is satisfied equally by a PR whose gate never executed.

Extends the note to release automation, where the same hole exists one level
up: a skipped or absent required context satisfies 'not failing', so the
dispatch condition has to require conclusion == success.
@proggeramlug proggeramlug changed the title docs(contributing): do not cancel the CI run of the PR being merged docs(contributing): do not cancel the CI run of the PR being merged (4 PRs through this hole in one day) Aug 31, 2026
@proggeramlug

Copy link
Copy Markdown
Contributor Author

Merged.

I am the proximate cause of half the incidents this documents, so let me confirm them rather than merge quietly past them.

#9169 is mine. I merged it after a local perry-runtime suite (2867 passing), the full perry-codegen suite, 60/60 lint gates, and an iterator probe. It broke main for four and a half hours. The probe is where it went wrong, and the reason generalises: I covered for…of, manual next() to exhaustion, spread, Array.from, Map/Set/String iterators, an OWN next shadowing the prototype thunk, matchAll, and getPrototypeOf(iter).next — every one of which is a method that actually exists as an own or inherited property. #9169's early-out resolves methods by ordinary property lookup, so the shapes it breaks are precisely the ones that don't exist that way: the iterator helpers Perry synthesizes further down the dispatch tower (#2874). [...gen().map(f)] was never in my probe, and no amount of iterating harder over real properties would have found it.

#9215, #9230 and #9235 are also mine, and the missing-fragment half is accurate. All three touched crates/, all three merged with lint cancelled, none carried a fragment. #9255 is now recovering the #9235 one by hand — which is exactly the "invisible until the release notes are cut" cost this section describes, arriving on schedule.

The distinction in the last paragraph is the part I'd underline for anyone skimming: pr-gate: pass and "nothing is red" are different assertions, and only the first is worth anything. A gate that never ran is absent from the status list, so it satisfies "0 failing" just as well as a gate that passed. That is the same failure I have hit three separate ways this week — a --help invocation I mistook for a running test suite, a scanner whose regex silently stopped matching, and a probe that measured a stale binary. Absence of evidence reading as evidence, every time.

Also right, and worth resisting: "the gate logic itself is correct and does not need changing." gate already runs if: always() and treats cancelled as failure. Every incident here is a bypass of a working gate, and the tempting fix — hardening the gate — would spend effort on the one part that is not broken.

The one thing I would add if you take a follow-up: the release-automation hole named at the end deserves its own check rather than a sentence, because it is the same bug one level up and nobody reads a paragraph at 2am. A required context that is skipped or absent satisfies "not failing" there too.

@proggeramlug
proggeramlug merged commit e87ffb5 into PerryTS:main Aug 31, 2026
18 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant