Improve analyzer performance for bundles with many assets#723
Open
christiango wants to merge 3 commits into
Open
Improve analyzer performance for bundles with many assets#723christiango wants to merge 3 commits into
christiango wants to merge 3 commits into
Conversation
🦋 Changeset detectedLatest commit: d252e7a The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
There was a problem hiding this comment.
Pull request overview
This PR optimizes getViewerData() to avoid repeatedly scanning the full root compilation module list for each analyzed asset, improving report-generation performance for compilations with many assets while keeping the public API and chart-data schema unchanged.
Changes:
- Build a single deduplicated/root module list once and index it by chunk ID for faster per-asset module selection.
- For root assets, gather and deduplicate candidate modules from referenced chunks, then restore original module ordering via stored indices.
- Add a focused unit test covering mixed numeric/string chunk IDs, shared modules, unchunked modules, duplicate module IDs, and runtime modules; add a patch changeset entry.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/analyzer.js |
Implements chunk-indexed module selection to avoid per-asset full-module scans (keeps child-compilation handling on the existing path). |
test/analyzer.js |
Adds unit coverage ensuring multi-chunk module ordering and deduplication semantics remain correct. |
.changeset/calm-assets-match.md |
Records the performance improvement as a patch-level change. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
getViewerData()rebuilt the root compilation's module list and scanned every module once per analyzed asset. For ordinary root assets, this made module selection roughly O(assets × modules), even though every asset used the same compilation module list.No matching public issue or pull request was found for this report-generation bottleneck.
Solution
The public API and chart-data schema are unchanged.
Complexity and correctness
Root asset selection changes from repeatedly scanning the complete module list for every asset to a one-time O(module chunk memberships) index plus per-asset candidate gathering and O(k log k) ordering, where k is the number of candidate modules for that asset.
Indexing occurs after the existing runtime-module and duplicate module-ID filtering, so those semantics remain unchanged. Exact
Mapkeys preserve the distinction between numeric and string chunk IDs. Candidate deduplication handles modules shared across referenced chunks, while index sorting preserves the original global module order. Modules without chunks remain excluded from chunk-backed assets. Child-compilation matching is intentionally unchanged.Performance
Observed on one large workload (these measurements were not rerun as part of this pull request):
Peak memory remained around 11–12 GB; this change does not claim a memory improvement. Stats conversion, measured separately at 7.3–8.3 seconds, is outside this optimization.
A local synthetic check used 1,486 assets and 30,000 modules with one chunk assignment per module. Three sequential
getViewerData()runs measured 2,206.6, 2,189.5, and 2,174.8 ms before; 45.1, 43.0, and 32.0 ms after. This is an illustrative module-selection comparison, not a repository benchmark or a wall-time assertion.Tests
Added focused generated-chart-data coverage for an asset referencing numeric and string chunk IDs. The fixture deliberately traverses chunks in a different order from the global module list, shares a module across both chunks, includes an unchunked module, a duplicate module ID, and a runtime module. The assertion verifies original output order and single inclusion of the shared module.
Existing analyzer fixtures continue to cover modules embedded in chunks, multiple entrypoints, worker/child compilations, Webpack 5 single and multiple entries, and concatenated modules.
Validation
npm test -- --runTestsByPath test/analyzer.js --silent— 27 passed, 4 skippednpm test -- --silent— 129 passed, 4 skippednpm run lint— ESLint, TypeScript, and Prettier checks passednpm run build— analyzer and production viewer builds passednumeric.js,shared.js, andstring.jsappeared once in original module order