Skip to content

Add S3 Docker build cache support#338

Draft
WVerlaek wants to merge 1 commit intomainfrom
wk/docker-s3-build-cache
Draft

Add S3 Docker build cache support#338
WVerlaek wants to merge 1 commit intomainfrom
wk/docker-s3-build-cache

Conversation

@WVerlaek
Copy link
Member

Description

Add support for S3-backed Docker build layer caching using BuildKit's type=s3 cache backend. This enables faster Docker builds by caching intermediate layers in S3.

New environment variables:

  • LEEWAY_DOCKER_S3_CACHE_BUCKET - S3 bucket name (required)
  • LEEWAY_DOCKER_S3_CACHE_REGION - AWS region (required)
  • LEEWAY_DOCKER_S3_CACHE_PREFIX - Optional prefix for cache keys
  • LEEWAY_DOCKER_S3_CACHE_MODE - Cache mode: min or max (default: max)
  • LEEWAY_DOCKER_S3_CACHE_ENDPOINT - Custom S3 endpoint for S3-compatible storage

New CLI flags:

  • --docker-s3-cache-bucket
  • --docker-s3-cache-region
  • --docker-s3-cache-prefix
  • --docker-s3-cache-mode
  • --docker-s3-cache-endpoint

When S3 cache is enabled, leeway automatically uses docker buildx build with --cache-from and --cache-to flags pointing to the configured S3 bucket.

Related Issue(s)

Fixes CORE-

How to test

# Via environment variables
export LEEWAY_DOCKER_S3_CACHE_BUCKET=my-cache-bucket
export LEEWAY_DOCKER_S3_CACHE_REGION=us-east-1
export LEEWAY_DOCKER_S3_CACHE_PREFIX=docker-cache/
leeway build :my-docker-package

# Via CLI flags
leeway build \
  --docker-s3-cache-bucket=my-cache-bucket \
  --docker-s3-cache-region=us-east-1 \
  --docker-s3-cache-prefix=docker-cache/ \
  :my-docker-package

# Verify the build command includes cache flags
leeway build -v :my-docker-package 2>&1 | grep cache

Requirements:

  • Docker Buildx
  • AWS credentials configured (via environment variables, IAM role, or AWS config file)
  • S3 bucket with appropriate permissions

Add support for S3-backed Docker build layer caching using BuildKit's
type=s3 cache backend. This enables faster Docker builds by caching
intermediate layers in S3.

Configuration via environment variables:
- LEEWAY_DOCKER_S3_CACHE_BUCKET: S3 bucket name (required)
- LEEWAY_DOCKER_S3_CACHE_REGION: AWS region (required)
- LEEWAY_DOCKER_S3_CACHE_PREFIX: Optional prefix for cache keys
- LEEWAY_DOCKER_S3_CACHE_MODE: Cache mode (min/max, default: max)
- LEEWAY_DOCKER_S3_CACHE_ENDPOINT: Custom S3 endpoint

Also adds corresponding CLI flags for all options.

When S3 cache is enabled, leeway automatically uses docker buildx build
with --cache-from and --cache-to flags.

Co-authored-by: Ona <no-reply@ona.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant