fix(resolver): resolve TypeScript imports of workspace packages - #506
Conversation
A monorepo imports its own packages by name: `@calcom/lib/hooks/useLocale` means `packages/lib/hooks/useLocale.ts`. The extractor wrote the target dotted, `@calcom.lib.hooks.useLocale`, and resolve_import_target returned None for every TypeScript source, so the edge never reached the module. On cal.com 86% of IMPORTS edges hung on such names, and the impact graph of a module imported in 256 places was the module alone. The pipeline now reads the `package.json` files it walks past, maps each package name to the directory FQN the TypeScript extractor gives it, and hands that to the resolver. resolve_import_target gains a TypeScript branch of the same shape as the Python layout-prefix one: a known name, matched on a segment boundary, longest first, rewritten only onto a node that exists — otherwise the edge stays visibly unresolved. The repository root's manifest is not a workspace package, a name claimed by two directories is not mapped, and an unreadable manifest is skipped. The mapping is recorded in ingest_state, and a change to it rebuilds the graph: an incremental run re-resolves only changed files, so after a package rename an unchanged importer would otherwise keep its edge onto the old package. TypeScriptExtractor gains a public module_fqn, which parse itself now uses, so the package map names directories by the extractor's own rule rather than a copy. Package collection lives in cgis.workspaces: added to IngestionPipeline it tripped the god-object baseline. Measured against main on the workspace checkouts: cal.com IMPORTS resolved 14.5% -> 55.9%; still on @-names 56.1% -> 14.8% grafana IMPORTS resolved 37.5% -> 59.4%; still on @-names 29.0% -> 7.1% useLocale impact graph: 1 node, 0 edges -> 289 nodes, 463 edges Not covered: tsconfig `paths` aliases, and `.d.ts` modules, whose FQN keeps a `.d` suffix so `@calcom/types/Calendar` still misses `packages.types.Calendar.d`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces support for resolving TypeScript workspace package imports in monorepos by parsing package.json manifests during the repository walk. It adds a new WorkspacePackages class to collect package mappings, updates the ResolverEngine and SymbolIndex to map TypeScript imports to their corresponding workspace modules, and persists these mappings in the SQLite store to trigger graph rebuilds when packages change. Feedback on the changes suggests avoiding a top-level import of TypeScriptExtractor in src/cgis/workspaces.py to prevent potential ImportError issues in environments lacking TypeScript dependencies, recommending instead a dynamic check of the class name in the MRO.
| from cgis.extractors.base import BaseExtractor | ||
| from cgis.extractors.typescript_extractor import TypeScriptExtractor |
There was a problem hiding this comment.
To prevent potential ImportErrors in environments where TypeScript dependencies (like tree-sitter-typescript) are not installed, avoid importing TypeScriptExtractor at the top level of this module. Since pipeline.py always imports workspaces.py, any top-level import of TypeScriptExtractor will trigger the import of tree_sitter_typescript even during Python-only runs.
| from cgis.extractors.base import BaseExtractor | |
| from cgis.extractors.typescript_extractor import TypeScriptExtractor | |
| from cgis.extractors.base import BaseExtractor |
There was a problem hiding this comment.
Fixed in 1171e1d — the observation is right, though not the stated consequence.
Verified in a fresh interpreter: importing cgis.pipeline did not load tree_sitter_typescript on main and did on this branch, because pipeline.py always imports workspaces.py. So the generic pipeline had gained a dependency on one language's extractor.
An ImportError was not reachable, though: tree-sitter-typescript is in the required dependencies in pyproject.toml, not an extra. The real cost was the layering, plus an unneeded load on Python-only runs.
workspaces.py now imports only extractors.base. New test test_importing_the_pipeline_does_not_load_a_language_grammar runs the import in a subprocess and asserts the grammar stays unloaded — it failed with True before the change.
| self._extractor = next( | ||
| (e for e in extractors.values() if isinstance(e, TypeScriptExtractor)), None | ||
| ) |
There was a problem hiding this comment.
Dynamically check for the TypeScriptExtractor class name in the MRO to avoid a hard dependency on the TypeScriptExtractor import at the top level of this module.
self._extractor = next(
(
e
for e in extractors.values()
if any(cls.__name__ == "TypeScriptExtractor" for cls in type(e).__mro__)
),
None,
)There was a problem hiding this comment.
Addressed in 1171e1d, by a different route than the one suggested.
Matching cls.__name__ == "TypeScriptExtractor" across the MRO would work today, but it fails silently if the class is ever renamed, and mypy cannot see that the selected object has module_fqn, so the call site would need a cast.
Instead there is a @runtime_checkable ModuleNamer protocol in extractors/base.py, and WorkspacePackages takes the extractor registered for .ts by extension and keeps it only if isinstance(extractor, ModuleNamer). Selecting by extension is also what the map actually needs — the .ts naming rule — rather than whichever TypeScript-derived class appears.
Review found that workspaces.py imported TypeScriptExtractor at module level, and pipeline.py always imports workspaces.py. Verified: importing cgis.pipeline did not load tree_sitter_typescript on main and did on this branch — the generic pipeline had gained a dependency on one language's extractor. The fix does not match on the class name, as suggested: a string compare breaks silently on a rename and hides module_fqn from mypy. Instead a runtime-checkable ModuleNamer protocol lives in the language-agnostic extractors.base, and WorkspacePackages takes the extractor registered for `.ts` by extension. A test runs the import in a fresh interpreter and asserts the grammar stays unloaded. tree-sitter-typescript is a required dependency, so no ImportError was reachable in a correct install; the cost was the layering and an unneeded load for Python-only runs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|



Refs #504 — covers workspace packages; tsconfig
pathsand.d.tsmodules remain (below).Problem
A monorepo imports its own packages by name:
@calcom/lib/hooks/useLocalemeanspackages/lib/hooks/useLocale.ts. The extractor wrote the target dotted andresolve_import_targetreturnedNonefor every TypeScript source, so the edge never reached the module. On cal.com 86% ofIMPORTSedges hung on such names, and the impact graph ofuseLocale— called from 318 places — was the module alone. For Guardian that meant the graph arm could not say who depends on a change.Change
cgis.workspaces.WorkspacePackagescollects thepackage.jsonfiles the pipeline walks past into package name → directory FQN. The repository root's manifest is not a workspace package; a name claimed by two directories is not mapped; an unreadable manifest is skipped with a warning.resolve_import_targetgains a TypeScript branch of the same shape as the Python layout-prefix one (resolver: module IMPORTS targets never go through layout reconciliation (31 of 6,293 resolve) #494): a known name, matched on a segment boundary, longest name first, rewritten only onto a node that exists. Anything else stays visibly unresolved.ingest_state, and a change to it rebuilds. An incremental run re-resolves only changed files, so after a package rename an unchanged importer would otherwise keep its edge on the old package —test_renaming_a_package_rebuilds_so_an_unchanged_importer_followsfails without the rebuild.TypeScriptExtractor.module_fqnis public and used byparseitself, so the map names directories by the extractor's own rule rather than a copy.Package collection started inside
IngestionPipelineand trippedtest_god_object_baseline_not_exceeded; it lives in its own module now. Accessing_pick_source_rootfrom the pipeline tripped SLF001, which is whatmodule_fqnreplaces.Measured
Re-ingested the workspace checkouts with this branch and with
main(viagit archive, same venv):@…nameget_impact_graph("packages.lib.hooks.useLocale", 2): 1 node, 0 edges → 289 nodes, 463 edges.Python sources are untouched by construction (
test_a_python_source_is_not_rewritten); sentry was not re-measured.Not covered
paths(@lib/*,@components/*— 181 imports on cal.com). Scoped per tsconfig withextendschains; a separate change..d.tsmodules keep a.dsuffix in their FQN —packages/types/Calendar.d.ts→packages.types.Calendar.d— so@calcom/types/Calendarstill misses. Most of the 288 remaining@calcom.*imports are this. Fixing it renames existing node ids, so it is its own change.Verification
make format && make lint && make type-check && make pytest && make doc-coverage: clean — 2710 passed, mypy strict clean, docstring coverage 99.3%. Pre-commit hooks pass.🤖 Generated with Claude Code