From 797b94f0ff49c8f4117dc0f4720967dffa42c2dc Mon Sep 17 00:00:00 2001 From: Aaron Queen Date: Wed, 14 Oct 2026 02:16:12 -0600 Subject: [PATCH] perf(kernel): resolve a Python class's bases once, not once for every scan over the class python_descendant_members resolves the bases of every same-named method's class for each requested class, so a sync that retries two or three references of the form Class.__init__ resolves the same 2,567 classes' bases again for each. A class's bases depend only on its own header and file; they are now kept per class for the life of the resolver. --- CHANGELOG.md | 1 + codegraph-kernel/src/resolve/member_fn_ref.rs | 4 ++++ codegraph-kernel/src/resolve/mod.rs | 4 ++++ 3 files changed, 9 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7fb0db2095..ddb0626620 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -100,6 +100,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### Fixes - **Syncing a changed file in a large Python project is faster.** When a Python class has no method of the name being linked (a test class that passes `Handler.__init__` as a value, say), the resolver checks every class with that method for one that derives from it. For each class whose base is a package import (`import unittest`, then `unittest.TestCase`) it parsed the whole file again, and its cache holds only a few files, so two such references took about a second each. A plain one-line `import a.b` is now read from its line; other forms still use the parse. In CPython a one-file sync takes a median 5.5 s instead of 7.7 s on Node and 5.2 s instead of 8.1 s on Bun (4 alternating runs per build, ranges not overlapping; the phase that retries unresolved references drops from 4.6 s to 2.3 s). A full index is within the run-to-run variation, and the graph is the same on CPython and Django (no edge differs). +- **The same sync resolves each Python class's bases once.** The scan above resolved the base classes of every candidate again for each reference that needed it. The bases of a class depend only on its own header and file, so they are now kept for the rest of the run. In CPython the phase that retries unresolved references drops from a median 2.29 s to 2.08 s on Node (-9%, 4 alternating runs per build, ranges not overlapping) and a one-file sync by about 3% (5.33 s to 5.15 s on the second sync); on Bun the sync time shows no clear change. A full index is within the run-to-run variation, and no edge, node or unresolved reference differs on CPython or Django. - **Questions to the MCP server are faster in a project with no `codegraph.json`.** While ranking results, CodeGraph checked for the project's `codegraph.json` once per result path, and with no such file each check raised and caught an error. It now asks the file system without raising, so `codegraph_explore` and the other query tools spend less CPU on most projects. Editing, adding or deleting `codegraph.json` is still picked up without a restart (#491). - **Indexing a project with a bundled minified JavaScript file is faster.** For each reference in a one-line file such as `jquery.min.js`, the resolver counted characters from the start of the line to find the reference's column, many times per reference. It now skips straight to the column when the text before it is ASCII. Alamofire, whose docs folder carries jQuery, resolves its references in a median 0.93 s instead of 1.25 s (profiling runs on a busy host, 5,614 nodes and 13,673 edges either way). Separately, the resolver now looks nodes up by query only when there are fewer than 15,000 pending references and fewer than 0.3 per node; at either limit it loads the node table. A full index of a small project such as gin (about 4.4 references per node) therefore loads the table, saving about 60 ms of its one-second index, while a one-file sync still queries. In the 2026-10-12 benchmark on Node, a full index of Alamofire takes 1.81 s against upstream's 1.73 s and gin 1.15 s against 1.13 s ([results](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md)). - **Indexing a project with long minified lines that contain a non-ASCII character is faster.** The resolver turns a reference's column into a byte offset many times per reference, and when any character before the column was not ASCII it walked every character on the line each time. It now counts a 64-byte block at a time and walks characters only inside the block that holds the column. Indexing JetBrains Exposed, whose docs folder carries minified scripts, takes a median 25.7 s instead of 37.0 s (5 alternating runs per build, baseline against baseline within 0.3%; 28,689 nodes and 82,842 edges either way). nocodb, with no long lines, is unchanged within the run-to-run band (6.53 s against 6.74 s). diff --git a/codegraph-kernel/src/resolve/member_fn_ref.rs b/codegraph-kernel/src/resolve/member_fn_ref.rs index 6e46997b2c..c205174195 100644 --- a/codegraph-kernel/src/resolve/member_fn_ref.rs +++ b/codegraph-kernel/src/resolve/member_fn_ref.rs @@ -361,6 +361,9 @@ impl KernelResolver { /// The classes a Python class header names as bases, each resolved from /// the class's own file. pub(super) fn python_bases(&mut self, cls: &KNode, r: &ResolveRefIn) -> Res>> { + if let Some(hit) = self.py_bases_memo.get(&cls.id) { + return Ok(hit.as_ref().clone()); + } let Some(lines) = self.read_file(&cls.file_path) else { return Ok(Vec::new()); }; @@ -371,6 +374,7 @@ impl KernelResolver { out.push(base); } } + self.py_bases_memo.insert(cls.id.clone(), Rc::new(out.clone())); Ok(out) } diff --git a/codegraph-kernel/src/resolve/mod.rs b/codegraph-kernel/src/resolve/mod.rs index fd41ce35d8..a4ff15216f 100644 --- a/codegraph-kernel/src/resolve/mod.rs +++ b/codegraph-kernel/src/resolve/mod.rs @@ -672,6 +672,9 @@ pub struct KernelResolver { /// Python methods named `member` on classes deriving from a class, by /// (class id, member). py_descendants_memo: HashMap<(String, String), Rc>>>, + /// A Python class's resolved bases, by class id: they depend on the class's + /// own header and file, so every scan over the same classes shares them. + py_bases_memo: HashMap>>>, /// Assignments and annotations typing a Python class's `self.`, /// by (class id, field). py_field_facts_memo: HashMap<(String, String), Rc>>, @@ -875,6 +878,7 @@ impl KernelResolver { kotlin_scope_memo: HashMap::new(), kotlin_supers_memo: HashMap::new(), py_descendants_memo: HashMap::new(), + py_bases_memo: HashMap::new(), py_field_facts_memo: HashMap::new(), py_globals: Default::default(), ruby_ancestry_memo: HashMap::new(),