Skip to content

Import .wpress archives with entries larger than 2 GiB - #4797

Open
samuelmbabhazi wants to merge 1 commit into
Automattic:trunkfrom
samuelmbabhazi:fix/wpress-large-archives
Open

Import .wpress archives with entries larger than 2 GiB#4797
samuelmbabhazi wants to merge 1 commit into
Automattic:trunkfrom
samuelmbabhazi:fix/wpress-large-archives

Conversation

@samuelmbabhazi

Copy link
Copy Markdown

Related issues

How AI was used in this PR

I used an AI assistant to help trace the crash, draft the position-based reader and the sparse-file test fixtures. I reviewed every line of the change, ran the full CLI unit suite locally, and reproduced the original native assertion against trunk with the new tests before applying the fix. Reviewers may want to focus on the position arithmetic in readHeader/readBlockToFile and on the new error paths for damaged archives.

Proposed Changes

Importing a .wpress backup that contains an entry of 2 GiB or more crashed Studio with Assertion failed: args[3]->IsInt32(). Listing the archive skipped every entry by reading its whole content into a buffer sized from the entry header, and FileHandle.read() only accepts a length that fits in a signed 32-bit integer. The same read happened when a blocked traversal entry was skipped during extraction.

  • Entry content is never read or allocated just to be skipped anymore, whatever its size, so backups holding large media files or database dumps import again.
  • Damaged archives now fail clearly instead of silently or forever: an archive that ends before the size declared in a header raises a "truncated" error rather than looping on zero-byte reads, a read that returns fewer bytes than requested only forwards the bytes read, and read or write failures are reported instead of being swallowed, so a partial import is no longer presented as a success.
  • An archive whose trailing EOF marker is cut short is still accepted, as before, since every entry in it is complete.

Testing Instructions

  1. Run npm test -- --project cli backup-handler-wpress. The suite builds sparse archives, so an entry above 2 GiB costs a few kilobytes on disk. It covers listing and extraction with such an entry, byte-for-byte integrity across chunk boundaries, empty entries, archives cut inside an entry or a header, an EOF marker cut short, a non-numeric size field and a write failure.
  2. To see the original failure, restore trunk's apps/cli/lib/import-export/import/handlers/backup-handler-wpress.ts and run the same tests: the vitest worker dies with Assertion failed: args[3]->IsInt32() in node::fs::Read (Node 24.18).
  3. Manual check: create a .wpress export with All-in-One WP Migration from a site holding a file larger than 2 GiB under wp-content/uploads, then import it with studio site import or from the app. Before: the import crashes. After: the import completes and the large file is extracted intact.

Also passing locally: the full CLI unit suite (npm test -- --project cli --tagsFilter='!e2e', 101 files, 1192 tests), npm run typecheck and npm run lint for apps/cli.

Pre-merge Checklist

  • Have you checked for TypeScript, React or other console errors?

Listing a .wpress archive skipped each entry by reading its whole content
into a buffer sized from the header. `FileHandle.read()` only accepts a
length that fits in a signed 32-bit integer, so an entry of 2 GiB or more
killed the process with the native assertion `args[3]->IsInt32()` in
`node::fs::Read`. The same read happened when a traversal entry was
skipped during extraction.

Every read now uses an explicit position and the reader advances past an
entry with arithmetic only. Entry content is never allocated to be
discarded, whatever its size.

While there, the extraction loop is made safe against damaged archives:
a read that returns fewer bytes than requested only forwards what was
read, an archive that ends before the declared size raises a clear
"truncated" error instead of looping forever, and read or write failures
are reported instead of being swallowed.

Tests cover an archive with a sparse entry above 2 GiB (listing and the
skip path), byte-for-byte integrity across chunk boundaries, empty
entries, archives cut inside an entry or a header, a non-numeric size
field, and a write failure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Importing .wpress backup >2GB crashes Studio with Node fs.read Int32 assertion

1 participant