Import .wpress archives with entries larger than 2 GiB - #4797
Open
samuelmbabhazi wants to merge 1 commit into
Open
Import .wpress archives with entries larger than 2 GiB#4797samuelmbabhazi wants to merge 1 commit into
samuelmbabhazi wants to merge 1 commit into
Conversation
Listing a .wpress archive skipped each entry by reading its whole content into a buffer sized from the header. `FileHandle.read()` only accepts a length that fits in a signed 32-bit integer, so an entry of 2 GiB or more killed the process with the native assertion `args[3]->IsInt32()` in `node::fs::Read`. The same read happened when a traversal entry was skipped during extraction. Every read now uses an explicit position and the reader advances past an entry with arithmetic only. Entry content is never allocated to be discarded, whatever its size. While there, the extraction loop is made safe against damaged archives: a read that returns fewer bytes than requested only forwards what was read, an archive that ends before the declared size raises a clear "truncated" error instead of looping forever, and read or write failures are reported instead of being swallowed. Tests cover an archive with a sparse entry above 2 GiB (listing and the skip path), byte-for-byte integrity across chunk boundaries, empty entries, archives cut inside an entry or a header, a non-numeric size field, and a write failure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
How AI was used in this PR
I used an AI assistant to help trace the crash, draft the position-based reader and the sparse-file test fixtures. I reviewed every line of the change, ran the full CLI unit suite locally, and reproduced the original native assertion against trunk with the new tests before applying the fix. Reviewers may want to focus on the position arithmetic in
readHeader/readBlockToFileand on the new error paths for damaged archives.Proposed Changes
Importing a
.wpressbackup that contains an entry of 2 GiB or more crashed Studio withAssertion failed: args[3]->IsInt32(). Listing the archive skipped every entry by reading its whole content into a buffer sized from the entry header, andFileHandle.read()only accepts a length that fits in a signed 32-bit integer. The same read happened when a blocked traversal entry was skipped during extraction.Testing Instructions
npm test -- --project cli backup-handler-wpress. The suite builds sparse archives, so an entry above 2 GiB costs a few kilobytes on disk. It covers listing and extraction with such an entry, byte-for-byte integrity across chunk boundaries, empty entries, archives cut inside an entry or a header, an EOF marker cut short, a non-numeric size field and a write failure.apps/cli/lib/import-export/import/handlers/backup-handler-wpress.tsand run the same tests: the vitest worker dies withAssertion failed: args[3]->IsInt32()innode::fs::Read(Node 24.18)..wpressexport with All-in-One WP Migration from a site holding a file larger than 2 GiB underwp-content/uploads, then import it withstudio site importor from the app. Before: the import crashes. After: the import completes and the large file is extracted intact.Also passing locally: the full CLI unit suite (
npm test -- --project cli --tagsFilter='!e2e', 101 files, 1192 tests),npm run typecheckandnpm run lintforapps/cli.Pre-merge Checklist