Fuse snapshot memo subtree walks into one traversal - #6748
Conversation
Greptile SummaryThis PR combines snapshot memo artifact collection into one subtree traversal. The main changes are:
Confidence Score: 4/5No additional blocking issue was found in the updated code.
|
| Filename | Overview |
|---|---|
| reflex/compiler/utils.py | Adds the fused artifact collector and uses it for non-passthrough snapshot memo compilation. |
| tests/units/compiler/test_compiler_utils.py | Adds equivalence coverage for artifact values, ordering, structural children, and component-valued props. |
| news/6748.performance.md | Documents the snapshot memo traversal optimization. |
Reviews (3): Last reviewed commit: "Appease pyright on subtree walk over Bas..." | Re-trigger Greptile
Merging this PR will improve performance by 55.53%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | test_var_access[mutable_list] |
19.1 ms | 9.1 ms | ×2.1 |
| ⚡ | Simulation | test_var_access[mutable_dict] |
32.5 ms | 20 ms | +62.13% |
| ⚡ | Simulation | test_evaluate_page_with_hooks[_stateful_page] |
5.2 ms | 4.8 ms | +10.25% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/reflex-compiler-perf-t8ztc9-15-snapshot-memo-walks (5e8f117) with claude/reflex-compiler-perf-t8ztc9-14-format-tag-bypass (e4eaa9c)2
Footnotes
-
8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
claude/reflex-compiler-perf-t8ztc9-14-format-tag-bypass(002a553) during the generation of this report, so 218c8e2 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8d47b987d7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| child_parts = [ | ||
| _collect_subtree_artifacts(child, in_prop_tree=in_prop_tree) | ||
| for child in component.children |
There was a problem hiding this comment.
Include VarData component artifacts in fused walk
This fused traversal only descends structural children here (and component props above), but Bare overrides the legacy _get_all_custom_code and _get_all_dynamic_imports to also visit components stored on its contents VarData. After the snapshot memo path replaces the legacy walkers with this helper, a memo body containing a component Var/Bare such as a conditional branch with a NoSSRComponent or a component that returns _get_custom_code() will omit that dynamic import or custom code from the generated memo module, even though the rendered JSX still references it. Please preserve the VarData component edge (or otherwise delegate to the override behavior) when fusing the walk.
Useful? React with 👍 / 👎.
| return {} | ||
|
|
||
|
|
||
| def _collect_subtree_artifacts( |
There was a problem hiding this comment.
this is what the new plugin-based compiler is already designed to do: reduce tree walks and collect the information from all components on a single descent. instead of implementing a smaller, more limited version of that, we should probably spin up a subcompiler with its own CompileContext/PageContext without the MemoizationPlugin in the stack and compile the subtree root as a "page", then collect the data from the page context. That way as the compiler develops, we don't have to maintain parallel functionality.
82b57dd to
86c42bd
Compare
Compiling a snapshot memo definition walked the (deep-copied) subtree four times - _get_all_hooks, _get_all_custom_code, _get_all_dynamic_imports, _get_all_imports - each independently recursing children and prop components and re-calling per-node getters like _get_components_in_props. _collect_subtree_artifacts gathers all four in a single traversal, combining each artifact at every node in the same order as its dedicated recursion (hooks stay structural-tree-only; prop-subtree custom code still lands before the node's own add_custom_code). The walk runs before render() so add_hooks side effects land first, same as the legacy hooks-walk-then-render order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jy8uHH11KircGa2MbTrR8g
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jy8uHH11KircGa2MbTrR8g
e4eaa9c to
002a553
Compare
86c42bd to
5e8f117
Compare
|
Closing after triaging the stack against current main:
The stack has been re-stitched around this PR: #6751 now bases directly on #6798's branch. The walk-fusion goal is still worth pursuing via the subcompiler approach. |
All Submissions:
Type of change
Changes To Core Features:
What this does
Compiling a snapshot memo definition (
compile_experimental_component_memo, non-passthrough branch) walked the deep-copied subtree four separate times —_get_all_hooks,_get_all_custom_code,_get_all_dynamic_imports,_get_all_imports— each independently recursing children and prop components, and re-calling per-node getters like_get_components_in_propsonce per artifact. Snapshot memos dominaterx.foreach-heavy pages, so these redundant traversals multiply.Fix
New
_collect_subtree_artifactsgathers all four artifacts in a single traversal. Correctness is order-sensitive (hooks/custom-code dict insertion order determines the emitted JS), so the fused walk reproduces each legacy recursion's exact per-node combination order:in_prop_treeflag mirrors_get_all_hooksskipping prop edges entirely);add_custom_code→ children (the interleaving_get_all_custom_codeuses);merge_parsed_imports.The fused walk runs before
render()soadd_hooksside effects (derived props) land first, matching the legacy hooks-walk-then-render order (the passthrough branch already collects imports before render).Scoping note
The ticket also covers the tag-hash walks (
_get_component_hash). Those are entangled with the known_get_all_hooks_internalshared-cache mutation bug (ENG-10108): today's hash output can depend on cache pollution from prior walks, so a fused hash walk can't be verified byte-identical until that bug is fixed. Deferred to a follow-up after ENG-10108; this PR takes the definition-compile side, which is pollution-free.Verification
add_custom_code, dynamic imports, component-typed prop slots, and nested prop-subtree children asserts the fused walk's four results equal the four legacy recursions including dict key order, and that prop-subtree hooks don't leak out.test_compile_*[_stateful_page]exercises snapshot memos viarx.foreach.Part of a compiler-performance series; tracked in Linear as ENG-10105.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Jy8uHH11KircGa2MbTrR8g
Generated by Claude Code