Skip to content

fix(ci): stop the layer-cache prune from being able to fail a build - #7252

Merged
waleedlatif1 merged 1 commit into
stagingfrom
ci/prune-step-must-never-fail-a-build
Aug 29, 2026
Merged

fix(ci): stop the layer-cache prune from being able to fail a build#7252
waleedlatif1 merged 1 commit into
stagingfrom
ci/prune-step-must-never-fail-a-build

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

The settle loop added in 4f651038f3 reads cur="$(total)". Composite steps run under bash -e -o pipefail, where an assignment takes its command substitution's exit status — so any failing buildctl du aborts the step and fails the build.

Reproduced directly:

set -e -o pipefail
total() { false | grep -iE '^total:' | tr -s ' ' ' '; }
echo "echo form: $(total)"   # survives
cur="$(total)"               # ABORTS, exit 1

That asymmetry is why it was invisible: the echo "$(total)" calls run first and survive, and only the assignment inside the settle loop trips it.

Not hypothetical. Deleting a sticky disk out from under a running job makes buildkitd panic inside cache.(*cacheManager).DiskUsage, and it took a Build AMD64 job down exactly this way:

panic: runtime error: invalid memory address or nil pointer dereference
github.com/moby/buildkit/cache.(*cacheManager).DiskUsage(...)

grep also exits 1 whenever du prints no Total: line, so a genuinely empty cache would have done it too.

Fix

total() now always returns 0. Cache hygiene must never be able to fail a deploy.

The prune's own failure was already handled correctly — it sits in an if condition, and a failing condition does not trip -e.

Verification

  • Reproduced the abort and confirmed the fix under the exact shell flags GitHub uses (bash -e -o pipefail): the settle loop now completes with a totally failing du.
  • shellcheck clean; actionlint identical to the staging baseline; bun run lint passes.

Type of Change

  • Bug fix

Testing

Shell-level reproduction of both the failure and the fix, as above.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

The settle loop added in 4f65103 reads `cur="$(total)"`, and composite steps
run under `bash -e -o pipefail`, where an assignment takes its command
substitution's exit status. So any failing `buildctl du` aborts the step and
fails the build. `echo "$(total)"` survives the same failure, which is why this
was invisible in the paths that ran first.

That is not hypothetical. Deleting a sticky disk out from under a running job
makes buildkitd panic inside cache.(*cacheManager).DiskUsage, and it took a
Build AMD64 job down that way. `grep` also exits 1 whenever du prints no Total
line, so an empty cache would have done it too.

A cache-hygiene step must never be able to fail a deploy, so `total()` now always
returns 0. The prune's own failure was already handled — it is an `if` condition,
and a failing condition does not trip `-e`.
@vercel

vercel Bot commented Aug 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 29, 2026 2:42am

Request Review

@greptile-apps

greptile-apps Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR prevents cache-usage reporting failures from aborting Docker build jobs under Bash errexit and pipefail behavior.

  • Makes the total() helper return successfully when buildctl du fails or emits no Total: line.
  • Documents why the fallback is required in the cache-prune settle loop.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness or security issues identified.

The fallback prevents the command-substitution assignment from triggering Bash errexit while preserving the prune command's existing success and failure handling.

Important Files Changed

Filename Overview
.github/actions/docker-build/action.yml Adds a deliberate success fallback to cache-usage reporting so ancillary cache hygiene cannot fail a deployment build.

Reviews (1): Last reviewed commit: "fix(ci): stop the layer-cache prune from..." | Re-trigger Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 4795d04 into staging Aug 29, 2026
27 checks passed
@waleedlatif1
waleedlatif1 deleted the ci/prune-step-must-never-fail-a-build branch August 29, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant