Skip to content

perf(visitor): speed up AST traversal -- 41% less time - #293

Open
dexhunter wants to merge 1 commit into
graphql-python:mainfrom
dexhunter:perf/visitor-stack
Open

dexhunter wants to merge 1 commit into
graphql-python:mainfrom
dexhunter:perf/visitor-stack

Conversation

@dexhunter

Copy link
Copy Markdown

TL;DR

  • Reduce time spent in visit when walking a GraphQL AST.
  • Use tuple stack entries, create edit lists only when needed, and skip checks for unchanged nodes.
  • The GitHub schema benchmark takes 25.01 ms → 14.75 ms, using 41.0% less time.
  • Native tests: PYTHONPATH=src python -m pytest tests -q passed 3,610 tests; 18 skipped.

Refs #292

What & Why

Each traversal created named stack entries and empty edit lists, even when visitors made no changes.
It also repeated type checks and checked action values after callbacks returned None.

Use plain tuples for stack entries and create edit lists only when a visitor changes a node.
Reuse saved array state and keep action checks inside the branch for non-None results.
The callback order, callback arguments, and immutable edit rules stay the same.

Validation

Situation Before After Change
GitHub schema, one visitor 25.01 ms 14.75 ms 41.0% less time
GitHub schema, 25 parallel visitors 178.03 ms 171.57 ms 3.6% less time
Large query pickle decoding 17.17 ms 17.13 ms No measurable change (0.2% less)

These are median times from the repository benchmarks on CPython 3.12.13, Linux x86-64, and an AMD EPYC 9R14.
Both versions ran sequentially on the same isolated benchmark slot, with at least 50 rounds per case.
Pickle decoding does not call visit; its 0.2% difference is within measurement variation and shows no measured slowdown.
The parallel case spends most of its time calling the 25 visitors, so the traversal change helps less.

Run these commands on the base commit and this change:

PYTHONPATH=src python -m pytest tests/benchmarks/test_visit.py tests/benchmarks/test_serialization.py::test_pickle_large_query_decode --benchmark-enable --benchmark-min-rounds=50
PYTHONPATH=src python -m pytest tests -q
python -m ruff check src/graphql/language/visitor.py
python -m ruff format --check src/graphql/language/visitor.py
python -m mypy src/graphql/language/visitor.py

The complete native suite passed: 3,610 passed, 18 skipped. Ruff and mypy passed for the changed module.
Tests used pytest 9.0.2 and pytest-benchmark 5.3.0; the installed pytest version had a benchmark-plugin setup error on both versions.

Why this is safe and easy to maintain

The change stays inside visit and uses ordinary tuple unpacking and conditional list creation.
It keeps the existing traversal loop, so the change remains small and easy to read.
Output and callback behavior stay unchanged, including replacements, removals, skips, and early stops.
The existing tests in tests/language/test_visitor.py cover traversal order, paths, ancestors, edits, and parallel visitors.

Scope

This changes Python traversal bookkeeping only, with no API changes or new dependencies.
The GraphQL.js traversal order and immutable edit behavior remain unchanged.

AI assistance

Codex gpt-6.1-sol generated candidates tracked with Weco Observe.
Codex also helped select the candidate, run automated checks, and prepare this description.

Weco share link: https://dashboard.weco.ai/share/Fg0y4rbDatKHfqBUeKkfsdtT1nwbnT1k

Use tuple stack entries and create edit lists only when a node changes.
Reuse array state and skip action checks for callbacks returning None.
Keep callback order, callback arguments, and immutable edits unchanged.

The native GitHub-schema visitor benchmark drops from 25.01 ms to
14.75 ms. All 3,610 native tests pass, with 18 skipped.

Co-Authored-By: Aiden <aiden@weco.ai>
@dexhunter
dexhunter requested a review from Cito as a code owner October 30, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant