Skip to content

perf(kernel): syncing a changed Python file resolves each class's bases once - #494

Merged
bompus merged 1 commit into
fork/consolidatedfrom
perf/python-descendants-via-extends
Oct 14, 2026
Merged

bompus merged 1 commit into
fork/consolidatedfrom
perf/python-descendants-via-extends

Conversation

@bompus

@bompus bompus commented Oct 14, 2026 •

Copy link
Copy Markdown
Owner

Syncing one changed file in CPython takes about 3% less on Node (5.33 s to 5.15 s on the second sync) and its retry phase 9–10% less (2.29 s to 2.08 s, 2.19 s to 1.96 s); Bun's sync time shows no clear change.

Metric, CPython (3,710 files), median of 4 runs per build Before After Change
Retry phase, first sync, Node 2.29 s 2.08 s 9% lower (ranges do not overlap)
Retry phase, second sync, Node 2.19 s 1.96 s 10% lower (ranges do not overlap)
One-file sync, second sync, Node 5.33 s 5.15 s 3% lower (ranges do not overlap)
One-file sync, first sync, Node 5.45 s 5.25 s no clear change (ranges overlap)
One-file sync, Bun (both syncs) 5.59 s, 5.41 s 5.74 s, 5.33 s no clear change (first sync 2.8% slower, inside the 3% band)
Retry phase, Bun 2.29 s, 2.20 s 2.04 s, 2.02 s 11% and 8% lower; ranges overlap on the first sync
Full index, Node / Bun 36.16 s / 39.59 s 34.92 s / 39.49 s no clear change
Full index, Django (guard, Node, 2 runs per build) 12.27 s 12.44 s no clear change

The small size is expected: this removes part of the two slow scans from the earlier fix (#493), not the scans themselves. Figures are from codegraph sync after appending one line to Lib/os.py on a fresh index, builds alternated B F F B B F F B; the preset rule (median gain above 3% and non-overlapping ranges) was declared before the runs.

What changes

python_bases resolves a class's base classes from its header and file. The scan that looks for a descendant of a class (see #493) calls it for every class that has a method of the name being linked, again for each such reference. A class's bases depend only on that class, so they are now kept by class id for the rest of the run (py_bases_memo, beside the existing py_descendants_memo). 8 lines of Rust.

Checks

  • The graph is unchanged: on CPython (167,929 nodes, 600,467 edges, 364,595 unresolved references) and Django (62,500, 185,361, 144,585), no edge row, node row or unresolved reference differs between the builds in either direction.
  • Full vitest suite on this head: 702 files, 8,952 passed, 39 skipped, 0 failed; no Native stack trace or Worker exited unexpectedly in the raw output. Kernel golden dumps unchanged.
  • README: checked "About this fork". The rows that compare the CPython one-file sync with upstream are still out of date in the fork's disfavor (see perf(kernel): read a plain Python import line instead of re-parsing the file for each package-import base #493); this change moves the fork's figure by about 3%, so they are unchanged and wait for the fork-against-upstream re-run. CHANGELOG entry added under Fixes.

Not covered: the other slow parts of the retry phase (the first use of the attribute-writer index, about 0.34 s, and the descendant scan itself); other Python corpora; Windows and macOS.


Summary by cubic

Caches each Python class's resolved base classes so the descendant scan during a sync resolves them once per class instead of once per reference. Syncing one changed file in CPython drops the retry phase by 9–10% on Node (2.29 s/2.19 s to 2.08 s/1.96 s) and the one-file sync by about 3% (5.33 s to 5.15 s); Bun shows no clear change.

Benchmarks (CPython, 3,710 files, median of 4 alternating runs per build)

Metric, Node Before After
Retry phase, first sync 2.29 s 2.08 s
Retry phase, second sync 2.19 s 1.96 s
One-file sync, second sync 5.33 s 5.15 s

A full index and Bun's one-file sync stay within run-to-run variation. This removes part of the slow scans from #493, not the scans themselves.

Verification

  • The graph is unchanged on CPython and Django: no edge, node, or unresolved reference differs.
  • Full vitest suite passes (8,952 passed, 0 failed) and kernel golden dumps are unchanged; a CHANGELOG entry is added under Fixes.

Written for commit 797b94f. Summary will update on new commits.

View guided diff Turn on auto-fix

Summary by CodeRabbit

  • Performance
    • Improved Python analysis performance, reducing CPython retry-phase time from 2.29 seconds to 2.08 seconds and one-file sync time from 5.33 seconds to 5.15 seconds in reported benchmarks.
    • Full-index timing remained within normal run-to-run variation, and no graph output differences were observed in CPython or Django benchmarks. No clear sync-time change was measured with Bun.

… scan over the class

python_descendant_members resolves the bases of every same-named method's class for each requested class, so a sync that retries two or three references of the form Class.__init__ resolves the same 2,567 classes' bases again for each. A class's bases depend only on its own header and file; they are now kept per class for the life of the resolver.
@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 90095988-3fc1-426a-b13a-f1b4d8d5dba6
📥 Commits

Reviewing files that changed from the base of the PR and between dfdf4a3 and 797b94f.

📒 Files selected for processing: 3
  • CHANGELOG.md
  • codegraph-kernel/src/resolve/member_fn_ref.rs
  • codegraph-kernel/src/resolve/mod.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

The resolver now caches resolved Python base classes by class ID for the duration of a run. The changelog reports benchmark results and states that graph results did not change.

Changes: Python base-class memoization

Layer / File(s) Summary
Add and use the base-class memo
codegraph-kernel/src/resolve/mod.rs, codegraph-kernel/src/resolve/member_fn_ref.rs, CHANGELOG.md
KernelResolver initializes a per-class memo. python_bases returns cached bases when available and stores resolved bases after a cache miss. The changelog reports benchmark results and unchanged-graph claims.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 797b9

This change reuses resolved Python bases within a resolution run. The resolver lifecycle prevents changed class bases from reusing a prior run’s cache, so no actionable merge-blocking risk remains.

Security architecture risk: 🔵 Low · up to 797b9

The cache is temporary, and normal indexing and synchronization paths discard it. No new permissions or external access were identified. Cache validity when callers retain an instance across intervening data changes remains unverified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The newly cached state is confined to an individual resolver instance and its inheritance lookups; the added memo is not a process-global or cross-project cache.

Trust Boundaries and Controls

  • inferred — The inspected change reuses existing source-reading and import/class lookup paths rather than granting new authority. No introduced execution or privilege transition was identified along this changed path.

Resilience and Maintainability Implications

  • observed — runPostExtract retains a kernel that was already open, and its cache-clearing routine does not reset native memos. Normal sync cleanup closes the kernel, but validity under atypical retention across mutations remains unverified; this is not a demonstrated PR-introduced security condition.
Pre-merge checks: Passed 6
Passed Passed checks: 6 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: caching Python class base resolution once per class during syncing.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Suppressions Explained Passed The pull request adds only a changelog entry and Python-base memoization. The changed Rust lines add a cache lookup, cache insertion, a field, and initialization. No changed line adds or broadens a li…
User-Visible Changes Documented Passed PASS. The pull request changes only Python base-resolution memoization and adds a CHANGELOG entry. The diff adds no CLI command or flag, MCP tool or argument, supported language or framework, agent ta…
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

🔧 LanguageTool
CHANGELOG.md

LanguageTool checks are incomplete because the per-file request limit of 5 was reached. Remaining text was skipped; findings from completed checks are retained.

Comment @coderabbitai help to get the list of available commands.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

View guided diff | Turn on auto-fix | Re-trigger cubic

@bompus
bompus merged commit 747775b into fork/consolidated Oct 14, 2026
5 checks passed
@bompus
bompus deleted the perf/python-descendants-via-extends branch October 14, 2026 09:27
github-actions Bot pushed a commit that referenced this pull request Oct 23, 2026
… classes do (colbymchenry#2828)

The interface-impl pass links a base class's or an interface's method
to each subclass's override, so a call through the base reaches the
method that runs. It took a class's methods, and its supertype, by
language: typescript and javascript were in its language set, and tsx
and jsx were not. A class in a .tsx or .jsx file got no override
edges, and neither did a .ts class whose base or interface was in a
.tsx file. The pass's own gate, the static/instance split of colbymchenry#2761
and the other JS-family lists already count tsx and jsx, and the
set's history (#494, colbymchenry#708) gives no reason for leaving them out.

tsx and jsx are now in the set, so these classes get the edges that
.ts and .js classes get, statics paired with statics.

Validation: the new test fails 4 of 4 on main 013b91c and passes 4
of 4 on the fix, and dropping tsx or jsx alone from the set fails the
tests for it. A/B against main on 11 repos: outline gains 262 override
edges in 42 base-subclass pairs (26 emails overriding BaseEmail's
subject, render and preview; editor nodes, marks and extensions
overriding Node, Mark and Extension) and lexical 27 in 12, grouped by
pair and sampled against the source, none pairing a static with an
instance method. Nodes and all other edges are unchanged. excalidraw,
social-app, insomnia, two .jsx apps and the controls (three, webpack,
flask, gin) are byte-identical: their .tsx classes extend React's or
other packages' classes. explore on outline now leads with the flow
BaseEmail.send -> render -> WelcomeEmail.render, where main finds
none. A sync that adds an interface method, adds an override and
drops a subclass's base matches a fresh index. The pass takes 65-108
ms on both arms, and index time, median of 3 interleaved runs on
lexical, outline, excalidraw and social-app, is +1.7% in total, noise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant