Tracking: Introduce proper trunk/backport workflow

[rami3l]

For now every rustup release is directly merged onto trunk and thus we might want to block patch releases from happening once a major/breaking change has landed (or vice versa). This is not very practical.

To keep and even further increase the bandwidth and momentum, an important change we would need to make to the current workflow would be to allow people from both inside and outside of the project to contribute whenever they want, and for the changes to land in the right release in a more flexible timeframe, so that we can introduce major changes even when we are waiting for patch releases. Thus the desire for a backport branch that collects all the bugfixes in this minor cycle, with proper CI and even CD (when the new release process is ready), from which we will force-push our stables.

Surely this needs more bandwidth, so it will probably not happen immediately, but I can see that this might be an important step in ensuring the project's long term health.

Tasks

• Merge the initial PR: ci(backport): add support for backporting #4774
  • Verify this change with the dummy PR: ci(backport): test workflow #4792
• Test this change thoroughly with more dummy PRs, even merging some of them to make sure everything is good:
  • In both the CI files and git itself, rename backport/* to release/* as suggested:
    • ci(backport): rename backport branches to release/* #4791
  • The conclusion job is identical to the main one.
  • Other pre-enqueue jobs are identical to the main ones.
  • The in-queue jobs are identical to the main ones.
  • The after-merge jobs are identical to the main ones modulo the deployment.
    • The deployment to GitHub Actions is kept.
    • All other deployments should not trigger.
• Enable the manual triggering of schedule-level CI.
  • ci(test): add workflow_dispatch trigger on par with schedule #4796
  • ci(backport): test workflow, pt. 2 #4799
• Delete the release/1.29-dummy branch and create the release/1.29 branch.
• Inform the infra team to enable merge queue and add branch protection rules (notably the force-push ban) for release/* branches. Technically impossible due to GitHub restrictions (Add branch protection and merge queue for rustup release branch team#2372 (comment)).
  • Add branch protection and merge queue for rustup release branch team#2372
• Update the dev guide to reflect the new workflow, including the manual triggering of CI, changed branching rules, changed stable status (can be force pushed instead of fast-forwarded), etc.
  • docs(dev-guide): update release process with new flow #4805

[ChrisDenton]

We could maybe ask if anyone from infra would help set up a stable branch (maybe a beta branch too?). Even if we can't make much use of it to start with, having something to build on would be useful.

[rami3l]

@ChrisDenton The current stable branch is a placeholder for "what we want to deploy as the next release" but the semantics is too ambiguous it seems to me: we reuse it for every beta and every stable, and there's no guarantee that at a given point in time what that would be. I'm thinking about branches like X.YY with proper CI so that we can at least have something to track.

[djc]

For other projects I usually have branches like 1.29.x once main has incompatible changes and a maintenance release needs to be made. But IMO there is not much point in making these branches proactively, since you can always make them when needed?

[ChrisDenton]

I thought the hard part was not creating the branches themselves (that's of course easy) but that all our infrastructure is based on the current setup (CI scripts, etc).

[rami3l]

I thought the hard part was not creating the branches themselves (that's of course easy) but that all our infrastructure is based on the current setup (CI scripts, etc).

@ChrisDenton Exactly. For now a reasonable next step for us seems to be that:

• Versioned backport branches get full CI both during backport PRs and upon push.

  • Unlike trunk where we also have the more scheduled "heavy CI runs", on versioned backport branches they are disabled.
  • Nonetheless, we should have a mechanism to trigger heavy CI on demand on those branches without having to use the config on other branches.

• Change the release process so that the stable branch only serves as a staging area for releases. This will notably include force-pushing from any writer to this repo.

[rami3l]

So this is my plan: upon the next incompatible change please feel free to tell me, I'll create a branch named backport/1.29 (feel free to suggest alternative prefixes) and I'll coordinate with the mods to put force-push bans on backport/* branches, and yes they should have merge queue enabled as well :)

Once that's done I'll take care of CI on those branches.

[rami3l]

Pseudo-main CI on release/* is running right now: https://github.com/rust-lang/rustup/actions/runs/23918294937

Notably, the GitHub Actions artifacts are uploaded as usual, but no other deployment steps has been triggered 🚀

Next up, I'll:

• Verify that the in-queue workflows are working identically to the occasions when we target trunk. Impossible, see the update in the original post.
• Investigate if it's practical to trigger "heavy" CI on release/* on demand.

Actually the latter part can wait for a bit, since we already have stable as the final resort if that doesn't work out. Thus, it only adds more confidence rather than blocking the backport flow from happening.