Repository navigation
Cache exports lookups and path patterns in module specifier generation - #64724
Draft
Gavin Kline (gwkline) wants to merge 3 commits into
Draft
Gavin Kline (gwkline) wants to merge 3 commits into
Gavin Kline (gwkline) wants to merge 3 commits into
Conversation
Generating a module specifier for a file inside a package with an "exports" map scans every entry of the map, resolving and comparing a candidate path for each, until the subpath that publishes the target file is found. The nodebuilder caches the final specifier per importing file, so declaration emit repeated the scan for every file that referenced the same target: on a build with many declaration files and dependencies with large exports maps it was most of the time spent on module specifiers and about eight percent of all allocations. The answer depends only on the parsed package.json, the package directory and name, the target file and the resolution mode, so hosts now keep a memo of it keyed by exactly those inputs. The compiler's Program owns one and its emit host forwards to it; the auto-import alias resolver owns one; hosts may return nil to opt out. Keying by the parsed package.json means a re-read file never serves a stale answer. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…kups Lookups through a tsconfig "paths" (or package "typesVersions") table resolved every pattern against the table's base directory and recomputed the candidate specifiers for the target file once per pattern, although neither depends on the other. With a table of a few hundred entries, as in workspaces that map every library, this dominated declaration emit. - The resolved patterns of a table are now computed once per host and kept in the module specifier generation cache, keyed by the table and its base directory. The cache, previously only for package.json "exports" lookups, is renamed GenerationCache accordingly. - The candidate specifiers are computed once per lookup. - "exports" and "imports" lookups compute the target's emitted file names once per lookup instead of once per entry, and build an entry's specifier only when the entry matches. - "exports" lookups are also keyed by the compiler options, so a host serving several option sets never returns another set's answer; a new test with custom conditions failed without this. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GetVersionPaths returns VersionPaths by value and GetPaths builds the paths table lazily on the receiver, so every caller built a table of its own on its copy: module resolution through a package with typesVersions rebuilt it on every resolution, and module specifier generation, which now caches resolved patterns by table identity, would never have found a cached entry and would have kept adding new ones. Build the table once when the version paths are selected so that every copy shares it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Gavin Kline (@gwkline) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #64721
Analysis
When declaration emit (or auto-import) needs a specifier for a file, it searches the mapping tables that could name it:
exports. For a file in a package with an"exports"map,tryGetModuleNameFromExportswalks every entry and condition, resolving a path and comparing it with the target for each. For every string entry it also recomputes the target's JS file name (and, for"imports", its output and declaration file names throughoutputpaths). It also joins the package name with every subpath before knowing whether the entry matches.paths. With"paths"(or a package's"typesVersions"),tryGetModuleNameFromPathsresolves every pattern against the table's base directory. For every pattern it also recomputes the candidate specifiers of the target file, although they don't depend on the pattern andprocessEndingcan query the file system.The nodebuilder caches the final specifier per importing file, so both scans repeat for every file that names the same target. The cost is importing files × table size. Emit holds a file's checker exclusively, so this sits on the emit critical path.
Fix
ModuleSpecifierGenerationHostgainsModuleSpecifierGenerationCache(), returning amodulespecifiers.GenerationCacheor nil. The compiler'sProgramowns one, which its emit host forwards to, and the auto-import alias resolver owns another. The cache memoizes the subpath anexportsmap publishes a target under. The key is the parsed*packagejson.PackageJson, the package directory and name, the target file, the resolution mode, and the compiler options. A re-read package.json therefore never serves a stale answer, and neither does a different option set. Negative results are cached too.paths/typesVersionstable, keyed by the table and its base directory. A lookup becomes prefix and suffix compares against precomputed strings.tryGetModuleNameFromPathsare computed once per lookup.exports/importslookups compute the target's file names once per lookup, and build an entry's specifier only when that entry matches.typesVersionstables.GetVersionPathsreturnsVersionPathsby value, andGetPathsbuilt the table lazily on the receiver. Every caller therefore rebuilt the table on its own copy, including module resolution on every resolution through such a package, and the table had no stable identity to cache by. The table is now built once, inside thesync.Oncethat selects the version paths.TestGetVersionPathsSharesTablepins this.Results are unchanged: lookups return the same specifiers in the same table order, and emitted declarations are byte-identical on every project below. Reachability of
node_modulesis still checked outside the cached lookup, so this composes with #64599.New tests in
modulespecifiers/cache_test.goandpackagejson/packagejson_test.go:exportsscan, 180 subpaths, uncachedpathsscan, 400 patterns, uncachedpathsscan, 400 patterns, cachedexportsgenerator from the issuepathsgenerator from the issueThe profiling, the patch and this description were produced with Claude Code (Claude Fable 5.1 and Claude Opus 5.5). I have read and understand the change and will handle review myself.
Copilot Checklist
I successfully ran the applicable command at the end of my session, and it completed without error:
🤖 Generated with Claude Code