Skip to content

build upload: stream normalization + ZIP emission instead of buffering artifacts in memory #1497

Description

@jared-outpost

Background

sentry build upload normalizes every mobile build artifact fully in memory before assembling the deterministic upload wrapper:

  • normalizeIpa reads the entire IPA and re-emits it via a single zipSync.
  • normalizeBuildDirectory reads the whole XCArchive tree into memory.
  • collectDsymEntries extracts dSYM ZIPs to a temp dir but then reads the bytes back in, and the final zipSync assembles the wrapper as one buffer.

For large inputs (big XCArchives, large dSYM bundles) this holds the artifact plus the wrapper in memory at once, bounded only by Node's ~2 GiB buffer limit. This surfaced during review of #1467 (dSYM support), but it is not specific to dSYMs — it is the shared normalization path.

Proposed work

Convert the shared normalization + ZIP-emission path to stream for all build types:

  • Stage entries via temp files on disk instead of buffering.
  • Use a chunked/streaming ZIP writer instead of a single zipSync over an in-memory map.
  • Preserve the current deterministic wrapper output (STORE compression, fixed mtime, sorted entries) so chunk dedup across re-uploads is unaffected.

Scope notes

  • Cross-cutting change to packages/cli/src/lib/build/index.ts; touches IPA, XCArchive, APK/AAB, and dSYM paths.
  • Must keep byte-for-byte wrapper determinism (verified by the existing normalization tests).

Follow-up from #1467 (per review discussion).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions