Summary
setup-docker-builder v2 removed max-cache-size-mb
(#125).
As a workaround, we run a nightly workflow that mounts each v2 sticky disk and
prunes it with:
buildctl --addr tcp://127.0.0.1:1234 prune --all --keep-storage 10240
The prune completes successfully, but buildkitd v0.29.3-blacksmith panics when its disk-usage RPC is called afterward.
setup-docker-builder v2 also calls buildctl du in its post action, before it shuts down and commits the sticky disk. This makes the failure prevent the pruned sticky disk from being committed.
Impact
After the BuildKit panic:
- BuildKit's gRPC endpoint returns EOF.
- The v2 post action tries to shut down buildkitd.
- Its pkill -TERM buildkitd call can fail because the daemon has already exited.
- The action treats that as a cleanup failure.
- It skips sync, unmount, and sticky-disk commit.
The cache prune therefore reclaims space only on the transient attached disk; the pruned state is not persisted and the sticky disk continues to grow.
The GitHub Actions job still appear successful because the post action logs the cleanup failure without failing the job.
Behaviour
buildctl prune succeeds. Running buildctl du immediately afterwards terminates BuildKit:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0]
github.com/moby/buildkit/cache.(*cacheManager).DiskUsage
/src/cache/manager.go:1447
Afterward, buildctl returns:
error: failed to call diskusage: Unavailable: error reading from server: EOF
The v2 post action then logs:
error shutting down buildkitd process: Command failed: sudo pkill -TERM buildkitd
Cleanup failed: Command failed: sudo pkill -TERM buildkitd
Skipping sticky disk commit due to cleanup error
Observed environment
- Action: a5256a7 (v2.1.0)
- BuildKit: v0.29.3-blacksmith
- Runner: blacksmith-2vcpu-ubuntu-2404-arm
- Cache before prune: 146.93 GB
- Data reported as reclaimed by prune: 136.73 GB
Summary
setup-docker-builderv2 removedmax-cache-size-mb(#125).
As a workaround, we run a nightly workflow that mounts each v2 sticky disk and
prunes it with:
The prune completes successfully, but buildkitd v0.29.3-blacksmith panics when its disk-usage RPC is called afterward.
setup-docker-builder v2 also calls buildctl du in its post action, before it shuts down and commits the sticky disk. This makes the failure prevent the pruned sticky disk from being committed.
Impact
After the BuildKit panic:
The cache prune therefore reclaims space only on the transient attached disk; the pruned state is not persisted and the sticky disk continues to grow.
The GitHub Actions job still appear successful because the post action logs the cleanup failure without failing the job.
Behaviour
buildctl prune succeeds. Running buildctl du immediately afterwards terminates BuildKit:
Afterward, buildctl returns:
The v2 post action then logs:
Observed environment