Skip to content

feat(volume-backups): add gzip compression to volume backups - #5206

Open
anxo-outeiral wants to merge 1 commit into
Dokploy:canaryfrom
anxo-outeiral:feat/volume-backup-gzip-compression
Open

feat(volume-backups): add gzip compression to volume backups#5206
anxo-outeiral wants to merge 1 commit into
Dokploy:canaryfrom
anxo-outeiral:feat/volume-backup-gzip-compression

Conversation

@anxo-outeiral

@anxo-outeiral anxo-outeiral commented Aug 28, 2026

Copy link
Copy Markdown

What is this PR about?

Volume backups currently create uncompressed .tar archives using tar cvf. This means that for data that compresses well (SQLite databases, JSON, text logs, configuration files), the backup stored in S3 is the same size as the original volume — wasting storage space and increasing S3 costs.

This PR changes to tar czf (gzip compression), producing .tar.gz files that are significantly smaller for compressible data, reducing S3 storage costs and transfer time.

Changes

  • backup.ts: Changed tar cvftar czf and extension .tar.tar.gz
  • utils.ts: Updated retention filter --include "*.tar"--include "*.tar.gz" and error cleanup *.tar*.tar.gz

Compatibility

  • gzip is pre-installed in the ubuntu Docker image used for volume backups — no additional dependencies or runtime package installation needed
  • Restore is backward compatible: tar xvf in restore.ts auto-detects gzip compression, so existing restore logic continues to work for both old .tar and new .tar.gz files without any changes
  • Old .tar files already in S3 are not affected by the new retention policy (which now filters *.tar.gz); they will need manual cleanup

Testing

Tested locally with a full development stack (Dokploy dev server + MinIO as local S3 destination + Docker Swarm):

  • Verified gzip is available in the ubuntu Docker image
  • Created volume backup with gzip compression
  • Verified .tar.gz file appears in S3 destination (MinIO)
  • Verified file command reports gzip compressed data, from Unix
  • Restored from .tar.gz backup successfully
  • Verified keepLatestCount retention works with .tar.gz extension
  • Verified backward compatibility: tar xvf restores both .tar and .tar.gz

Compression results (13.4MB of text/JSON/SQL data):

Size
Volume (uncompressed) 13.4 MB
Backup in S3 (.tar.gz) 884 KB
Compression ratio ~15:1

Checklist

Before submitting this PR, please make sure that:

  • You created a dedicated branch based on the canary branch.
  • You have read the suggestions in the CONTRIBUTING.md file https://github.com/Dokploy/dokploy/blob/canary/CONTRIBUTING.md#pull-request
  • You have tested this PR in your local instance. If you have not tested it yet, please do so before submitting. This helps avoid wasting maintainers time reviewing code that has not been verified by you.

Issues related (if applicable)

Closes #3566

Screenshots (if applicable)

Backup log showing tar czf and .tar.gz:

Backup file name: gzip-test-vol-2026-08-28T15-31-31-553Z.tar.gz
Turning off volume backup: Yes
Starting volume backup
Volume backup done ✅
Starting upload to S3...
Upload to S3 done ✅
Local backup file cleaned up ✅

Greptile Summary

This PR gzip-compresses newly created volume-backup archives and updates current-format retention and failure cleanup patterns.

  • Changes generated volume archives from .tar to .tar.gz.
  • Enables gzip compression during archive creation.
  • Updates S3 retention and local error cleanup to match compressed archives.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking cleanup gap for local archives left by interrupted pre-upgrade backups.

Compression, listing, retention of new archives, and restore remain aligned, but the changed failure cleanup can leave legacy local .tar artifacts consuming disk space.

Files Needing Attention: packages/server/src/utils/volume-backups/utils.ts

Reviews (1): Last reviewed commit: "feat(volume-backups): add gzip compressi..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

Context used:

@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Aug 28, 2026
Comment on lines +159 to +160
// delete all the .tar.gz files
const command = `rm -rf ${volumeBackupPath}/*.tar.gz`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Legacy archives escape cleanup

If an interrupted pre-upgrade backup leaves a local .tar archive, the changed failure handler matches only .tar.gz, leaving the legacy archive on disk indefinitely and consuming server storage.

Knowledge Base Used: Backups and restore

Volume backups were creating uncompressed .tar archives using tar cvf.
This changes to tar czf (gzip compression), producing .tar.gz files
that are significantly smaller for compressible data (text, JSON, SQLite,
logs), reducing S3 storage costs.

Changes:
- backup.ts: tar cvf -> tar czf, extension .tar -> .tar.gz
- utils.ts: retention filter *.tar -> *.tar.gz, error cleanup updated

Compatibility:
- gzip is pre-installed in the ubuntu image used for volume backups
- tar xvf in restore auto-detects gzip, so existing restore logic
  works for both old .tar and new .tar.gz files
- Old .tar files in S3 are not affected by the new retention policy
  (which now filters *.tar.gz); they require manual cleanup

Closes Dokploy#3566
@anxo-outeiral
anxo-outeiral force-pushed the feat/volume-backup-gzip-compression branch from d01baef to 76615ad Compare August 28, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Supporting compression on volume backups

1 participant