improvement(ci): decouple image builds from the test gate#5701
Conversation
- build-amd64 now starts immediately and pushes only sha tags (ECR :sha, GHCR :sha-amd64). The EventBridge deploy triggers filter on exactly the latest/staging/dev ECR tags, so nothing deploys from these pushes. - New promote-images job retags sha -> latest/staging (and GHCR latest-amd64/version-amd64) via buildx imagetools once tests and migrations pass — a seconds-long manifest copy instead of rebuilding after the gate. Cuts push-to-deploy from ~13.5 to ~7 minutes. - Split the Next.js production build out of test-build into a parallel Build App job with its own sticky-disk keys, cutting PR feedback from ~5.5 to ~3.5 minutes. - create-ghcr-manifests and process-docs now gate on promote-images.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryHigh Risk Overview
Reviewed by Cursor Bugbot for commit f457ba0. Configure here. |
Greptile SummaryThis PR builds images in parallel with validation and promotes them only after the deployment gate passes. The main changes are:
Confidence Score: 5/5This looks safe to merge.
Important Files Changed
Reviews (2): Last reviewed commit: "improvement(ci): harden promotion — atom..." | Re-trigger Greptile |
…ate all mutable GHCR tags Review follow-ups: - promote-images is a single job (not a matrix): verifies all four :sha manifests exist before moving any deploy tag, so a missing image can't cause a partial mixed-version deploy - stale-run guard on promote-images and create-ghcr-manifests: re-running an old run no longer retags latest/staging back to stale code (a one-click prod rollback); superseded runs skip mutable tags with a warning while immutable sha/version tags still publish - ARM64 build now pushes only the immutable :sha-arm64 tag; latest-arm64 and version tags moved behind the gate into create-ghcr-manifests (closes the pre-existing hole where a failing run moved latest-arm64) - dropped dead detect-version needs from both build jobs - sticky-disk comment corrected (clone + last-writer-wins, not exclusive mounts); build job shares warm bun/node_modules disks, keeps its own turbo cache key so test/build entries don't evict each other
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit f457ba0. Configure here.
Problem
Push-to-production-deploy takes ~13.5 minutes because the pipeline is fully serial: tests (5m22) → migrate (0m46) → AMD64 image builds (4m48–7m01) → ECR tag push → CodePipeline. The builds only start after the gate clears, though nothing about building requires it. (Timings from run 29459378914.)
Design
Standard "build once, promote by retag" artifact promotion: images build in parallel with tests under immutable sha tags; passing the gate promotes the exact tested artifact by moving a tag (
docker buildx imagetools create, a seconds-long manifest copy) — never by rebuilding.Invariant: no mutable/documented tag moves, and nothing deploys, until tests + migrations pass. Only immutable sha tags exist before the gate.
ci.yml
build-amd64andbuild-ghcr-arm64start immediately and push only sha tags (ECR:<sha>, GHCR:<sha>-amd64/:<sha>-arm64). This also closes a pre-existing hole:latest-arm64used to be pushed before test results existed.promote-imagesjob (aftertest-build+migrate) retagssha → latest/stagingin ECR — the tag CodePipeline's EventBridge rules trigger on. Single job, not a matrix: it verifies all four sha manifests exist before moving any tag, so a partial mixed-version deploy can't happen.create-ghcr-manifests(after promote + arm64 build) now owns every mutable GHCR tag:latest-{amd64,arm64},latest, version tags, and the multi-arch sha manifest — all built from the immutable sha-arch tags.latestback to old code and trigger an immediate prod deploy of it. Now a superseded commit skips mutable-tag moves with a warning (immutable sha/version tags still publish, so releases can't be blocked by a racing push).test-build.yml
Tests (2m05) and
next build(2m26) were serial in one job; now parallel jobs. They share the warm bun-cache/node_modules sticky disks (identical content; Blacksmith disks are clone-on-mount + last-writer-wins commit, so concurrent mounts are safe) — but the build job gets its own Turbo cache key, since with a shared key the two jobs' new entries would evict each other every run.Verification
image-tag: ["latest"|"staging"|"dev"]) for all 9 pipelines — sha pushes trigger nothingactionlintclean (except pre-existing runner-label/SC2086-style notes shared with the old file)Expected impact
Accepted trade-off
Untested immutable sha-arch tags now exist on public GHCR for commits whose tests later fail (already true for arm64 today). They're plumbing tags nothing documents or deploys, but GHCR has no lifecycle expiry — if we care about accumulation, a scheduled
ghcruntagged/sha cleanup job is a small follow-up.Watch on merge
The push path (sha push + promote) first executes on the merge to staging — watch that run's
promote-imagesbefore this reaches main.