Skip to content

Improve analyzer performance for bundles with many assets#723

Open
christiango wants to merge 3 commits into
webpack:mainfrom
christiango:christiango-optimize-asset-module-matching
Open

Improve analyzer performance for bundles with many assets#723
christiango wants to merge 3 commits into
webpack:mainfrom
christiango:christiango-optimize-asset-module-matching

Conversation

@christiango

Copy link
Copy Markdown

Problem

getViewerData() rebuilt the root compilation's module list and scanned every module once per analyzed asset. For ordinary root assets, this made module selection roughly O(assets × modules), even though every asset used the same compilation module list.

No matching public issue or pull request was found for this report-generation bottleneck.

Solution

  • Build the deduplicated, runtime-filtered root module list once.
  • Index root modules by numeric or string chunk ID.
  • Gather only modules belonging to an asset's referenced chunks.
  • Deduplicate modules shared by multiple chunks and sort candidates by their original module-list index.
  • Keep child-compilation assets on the existing conservative full-scan path.

The public API and chart-data schema are unchanged.

Complexity and correctness

Root asset selection changes from repeatedly scanning the complete module list for every asset to a one-time O(module chunk memberships) index plus per-asset candidate gathering and O(k log k) ordering, where k is the number of candidate modules for that asset.

Indexing occurs after the existing runtime-module and duplicate module-ID filtering, so those semantics remain unchanged. Exact Map keys preserve the distinction between numeric and string chunk IDs. Candidate deduplication handles modules shared across referenced chunks, while index sorting preserves the original global module order. Modules without chunks remain excluded from chunk-backed assets. Child-compilation matching is intentionally unchanged.

Performance

Observed on one large workload (these measurements were not rerun as part of this pull request):

  • 1,486 analyzed JavaScript assets
  • about 8.58 MB serialized chart data and 8.89 MB static HTML
  • 25,620 compression operations over about 198 MB of input
  • analyzer report generation: 20.8–22.2 seconds before and about 6.2 seconds after the prototype
  • end-to-end raw samples: 100.63 and 93.40 seconds before; 74.46 and 73.61 seconds after
  • midpoint improvement: about 22.98 seconds / 23.7%

Peak memory remained around 11–12 GB; this change does not claim a memory improvement. Stats conversion, measured separately at 7.3–8.3 seconds, is outside this optimization.

A local synthetic check used 1,486 assets and 30,000 modules with one chunk assignment per module. Three sequential getViewerData() runs measured 2,206.6, 2,189.5, and 2,174.8 ms before; 45.1, 43.0, and 32.0 ms after. This is an illustrative module-selection comparison, not a repository benchmark or a wall-time assertion.

Tests

Added focused generated-chart-data coverage for an asset referencing numeric and string chunk IDs. The fixture deliberately traverses chunks in a different order from the global module list, shares a module across both chunks, includes an unchunked module, a duplicate module ID, and a runtime module. The assertion verifies original output order and single inclusion of the shared module.

Existing analyzer fixtures continue to cover modules embedded in chunks, multiple entrypoints, worker/child compilations, Webpack 5 single and multiple entries, and concatenated modules.

Validation

  • npm test -- --runTestsByPath test/analyzer.js --silent — 27 passed, 4 skipped
  • npm test -- --silent — 129 passed, 4 skipped
  • npm run lint — ESLint, TypeScript, and Prettier checks passed
  • npm run build — analyzer and production viewer builds passed
  • Generated a static multi-chunk report, opened it in a browser, and verified numeric.js, shared.js, and string.js appeared once in original module order

@changeset-bot

changeset-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: d252e7a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
webpack-bundle-analyzer Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@christiango christiango marked this pull request as ready for review July 10, 2026 18:43
Copilot AI review requested due to automatic review settings July 10, 2026 18:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes getViewerData() to avoid repeatedly scanning the full root compilation module list for each analyzed asset, improving report-generation performance for compilations with many assets while keeping the public API and chart-data schema unchanged.

Changes:

  • Build a single deduplicated/root module list once and index it by chunk ID for faster per-asset module selection.
  • For root assets, gather and deduplicate candidate modules from referenced chunks, then restore original module ordering via stored indices.
  • Add a focused unit test covering mixed numeric/string chunk IDs, shared modules, unchunked modules, duplicate module IDs, and runtime modules; add a patch changeset entry.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/analyzer.js Implements chunk-indexed module selection to avoid per-asset full-module scans (keeps child-compilation handling on the existing path).
test/analyzer.js Adds unit coverage ensuring multi-chunk module ordering and deduplication semantics remain correct.
.changeset/calm-assets-match.md Records the performance improvement as a patch-level change.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/analyzer.js Outdated
Copilot AI review requested due to automatic review settings July 10, 2026 19:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread src/analyzer.js
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants