Skip to content

feat(html): replace vis-network with vendored force-graph renderer - #3493

Open
jakob1379 wants to merge 1 commit into
Graphify-Labs:v8from
jakob1379:t3code/replace-pyvis-graph-renderer
Open

feat(html): replace vis-network with vendored force-graph renderer#3493
jakob1379 wants to merge 1 commit into
Graphify-Labs:v8from
jakob1379:t3code/replace-pyvis-graph-renderer

Conversation

@jakob1379

@jakob1379 jakob1379 commented Sep 11, 2026

Copy link
Copy Markdown

Summary

The old graph.html loaded vis-network from unpkg, capped at 5k nodes, and on this repo fell back to a 1,020-node community summary. Forcing the full graph blocked the page for about 30 s and produced an unreadable blob.

graph.html now inlines force-graph 1.51.4 (MIT, Canvas 2D, bundles d3-force). It opens offline and renders this repo's full 15.9k-node graph with live physics. Graph data is unchanged: same node ids, edges, communities and labels.

Preview

Screen recording of the new renderer on graphify's own repo (drag, pin, spotlight, presets, halo):

output

Why force-graph

Surveyed pyvis, ipysigma, gravis, plotly, bokeh, pyecharts, graphviz, cosmograph and d3graph on the Python side, and vis-network, sigma.js v3, cytoscape.js, raw d3-force, cosmos.gl, force-graph and AntV G6 on the JS side. Static SVG was rejected: past roughly 10k marks the DOM cost makes the page sluggish, and CSS-only interactivity cannot do search, spotlight or drag.

force-graph won on size (about 178 KB minified, one file, no CDN), active maintenance, built-in d3-force physics, canvas hit-testing and drag, and a small API to build the panel on. cosmos.gl is faster but WebGL-only with mobile caveats. sigma needs graphology plus a separate layout package.

What changed

  • Community-seeded start positions plus cohesion, separation and a halo force keep communities as clusters and push disconnected pieces to the rim.
  • Physics panel with graphify / forceAtlas2 / compact / spread presets, tooltipped sliders, Reheat / Unpin / Fit. Drag pins a node, right-click releases it, click spotlights its neighbours.
  • Sidebar panels collapse with an animation. Communities sits above Physics.
  • Viewport culling. Edges are hidden while big graphs settle and when zoomed far out.
  • MAX_NODES_FOR_VIZ raised to 20,000 and graphify export html reads it from the exporter instead of hardcoding 5000.
  • The vendored JS ships via [tool.setuptools.package-data] and is covered by the wheel packaging test.

Known trade-offs

  • Straight edges instead of vis-network's curves.
  • Output is about 13.6 MB for this repo (graph data plus the inlined renderer).
  • Background tabs throttle requestAnimationFrame, so the initial settle can stop early. Reheat restarts it.

Testing

  • uv run --frozen pytest tests/test_export.py tests/test_pipeline.py tests/test_cli_export.py tests/test_wheel_packaging.py: 249 passed
  • uv build --wheel confirmed graphify/exporters/vendor/force-graph.min.js ships
  • Emitted script blocks parse under node; manual check in a browser at 15.9k nodes

graph.html now inlines force-graph 1.51.4 (Canvas 2D, d3-force) instead
of loading vis-network from unpkg, so the file opens offline and renders
this repo's full 15.9k-node graph instead of hitting the 5k cap.

- live physics with community seeding, cluster cohesion/separation and a
  halo force that pushes disconnected pieces to the rim
- physics panel with graphify / forceAtlas2 / compact / spread presets,
  tooltipped sliders, reheat / unpin / fit; drag pins a node, right-click
  releases it, click spotlights neighbours
- collapsible sidebar panels with animated collapse
- viewport culling, edges hidden while big graphs settle or when zoomed
  far out
- MAX_NODES_FOR_VIZ raised to 20000; CLI export reads the limit from the
  exporter instead of hardcoding 5000
- vendored JS ships in the wheel via package-data and is covered by the
  wheel packaging test

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.

Formal verification. 1 change(s) alter behavior, breaking input(s) attached.

Behavior changes: \_html\_styles changes behavior, here is the input that shows it.

The verifier found a concrete input on which \_html\_styles behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.

Guarantee: This is a sound refutation from a mathematical proof (SMT solver, over a bounded input domain): a concrete input on which the two versions provably differ.


Graphify review — findings

Replaces the vis-network graph renderer with a self-contained force-graph vendor bundle, raising the browser viz cap from 5,000 to 20,000 nodes and driving that limit through _viz_node_limit() in both the CLI and HTML exporter. Rebuilds the sidebar as collapsible, animated .panel sections with a physics-control panel, and swaps the hull-drawing _hyperedge_script into _html_script, which now receives hyperedges directly alongside nodes, edges, and legend data.

Worth a look

  • Unescaped graph titles are rendered as HTML tooltipsgraphify/exporters/html.py · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Raw JSON is embedded in a script tag without escaping script terminatorsgraphify/exporters/html.py:113 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Community label inserted into HTML without escapinggraphify/exporters/html.py:599 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 817 functions depend on the 490 functions this change touches.

Health — this change adds coupling hotspots:

  • new: _rebuild_code() — 115 callers, 51 callees
  • new: dispatch_command() — 2 callers, 124 callees
  • new: to_html() — 16 callers, 11 callees
  • new: run_pipeline() — 8 callers, 13 callees
  • new: qr() — 12 callers, 8 callees
  • new: _stale_graph_sources() — 7 callers, 6 callees
  • new: watch() — 5 callers, 7 callees
  • new: _run_hook_guard() — 4 callers, 8 callees
  • …and 4 more — each is listed as a finding

Verification — 817 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 763 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

262 of 262 test file(s) selected (100%) via static blast radius.

Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.

  • tests/test_affected_cli.py — impact, full-run-safety
  • tests/test_affected_member_seed.py — full-run-safety
  • tests/test_agents_platform.py — impact, full-run-safety
  • tests/test_analyze.py — full-run-safety
  • tests/test_anthropic_custom_endpoint.py — full-run-safety
  • tests/test_antigravity_install.py — full-run-safety
  • tests/test_apm_fallback_version.py — full-run-safety
  • tests/test_architecture_doc.py — full-run-safety
  • tests/test_astro_extraction.py — full-run-safety
  • tests/test_astro_import_ids.py — full-run-safety
  • tests/test_atomic_canvas_export.py — full-run-safety
  • tests/test_atomic_version_stamp.py — full-run-safety
  • tests/test_atomic_writes.py — full-run-safety
  • tests/test_backend_extras.py — full-run-safety
  • tests/test_benchmark.py — full-run-safety
  • tests/test_benchmark_raw_graph.py — full-run-safety
  • tests/test_build.py — full-run-safety
  • tests/test_build_merge_hyperedges_and_prune.py — full-run-safety
  • tests/test_build_merge_shrink_guard.py — full-run-safety
  • tests/test_builtin_global_type_refs.py — full-run-safety
  • tests/test_cache.py — full-run-safety
  • tests/test_callflow_html.py — full-run-safety
  • tests/test_cargo_introspect.py — full-run-safety
  • tests/test_carried_hyperedge_remap.py — full-run-safety
  • tests/test_case_sensitive_resolution.py — full-run-safety
  • tests/test_charmap_encoding.py — full-run-safety
  • tests/test_chunking.py — full-run-safety
  • tests/test_cjs_module_extension.py — full-run-safety
  • tests/test_claude_cli_backend.py — full-run-safety
  • tests/test_claude_md.py — full-run-safety
  • tests/test_cli_broken_pipe.py — full-run-safety
  • tests/test_cli_export.py — full-run-safety
  • tests/test_cli_help.py — full-run-safety
  • tests/test_cluster.py — full-run-safety
  • tests/test_codebuddy.py — impact, full-run-safety
  • tests/test_community_hub_labels.py — full-run-safety
  • tests/test_community_labels_skill.py — full-run-safety
  • tests/test_confidence.py — full-run-safety
  • tests/test_corrupt_graph_json.py — full-run-safety
  • tests/test_cpp_nested_and_cli.py — full-run-safety
  • tests/test_cpp_objc_cross_file_calls.py — full-run-safety
  • tests/test_cpp_preprocess.py — full-run-safety
  • tests/test_cross_extension_reexport_self_cycle.py — full-run-safety
  • tests/test_cross_language_call_resolution.py — full-run-safety
  • tests/test_cross_repo_member_calls.py — full-run-safety
  • tests/test_cross_repo_shared_types.py — full-run-safety
  • tests/test_csharp_call_site_generic_args.py — full-run-safety
  • tests/test_csharp_enum_members.py — full-run-safety
  • tests/test_csharp_field_generic_args.py — full-run-safety
  • tests/test_csharp_generic_callsites.py — full-run-safety
  • … and 212 more

non-code file(s) changed (README.md, pyproject.toml) → running the full suite for safety (a code graph can't see config/fixture/data deps)

changed code file(s) with no mapped test (README.md, graphify/exporters/vendor/force-graph.min.js, pyproject.toml) — a coverage gap or a missing link — running the full suite rather than only the selected tests

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

Behavior changes: \_html\_styles changes behavior, here is the input that shows it.

The verifier found a concrete input on which \_html\_styles behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.

Guarantee: This is a sound refutation from a mathematical proof (SMT solver, over a bounded input domain): a concrete input on which the two versions provably differ.

Could not verify: Could not verify build.

The verifier did not have enough to check build, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 9 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly AttributeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify build\_merge.

The verifier did not have enough to check build\_merge, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `graph_path` is annotated `str | Path | None` — outside the synthesizable primitive/collection set

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_is\_noise\_dir.

The verifier did not have enough to check \_is\_noise\_dir, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `parent` is annotated `'Path | None'` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_html\_script.

The verifier did not have enough to check \_html\_script, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: signature changed: no shared positional arity

No difference found (not proven): No behavior difference found in to\_html (not a proof).

The verifier ran both versions of to\_html on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify extract.

The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_import\_php.

The verifier did not have enough to check \_import\_php, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 200 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly AttributeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify extract\_bash.

The verifier did not have enough to check extract\_bash, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_extract\_generic.

The verifier did not have enough to check \_extract\_generic, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_apply\_symbol\_resolution\_facts.

The verifier did not have enough to check \_apply\_symbol\_resolution\_facts, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_collect\_python\_symbol\_resolution\_facts.

The verifier did not have enough to check \_collect\_python\_symbol\_resolution\_facts, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify extract\_rust.

The verifier did not have enough to check extract\_rust, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify extract\_sql.

The verifier did not have enough to check extract\_sql, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

No difference found (not proven): No behavior difference found in \_pinned\_python (not a proof).

The verifier ran both versions of \_pinned\_python on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify install.

The verifier did not have enough to check install, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

Could not verify: Could not verify generate\_community\_labels.

The verifier did not have enough to check generate\_community\_labels, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 1 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

· 12 more finding(s) on lines outside this diff (see the check run).

@jakob1379

Copy link
Copy Markdown
Author

Checked the three advisory findings against 0679681 by feeding </script><img src=x onerror=alert(1)> through to_html as a node label, an edge relation, and a community name. All three are false positives. The escaping is present and unchanged from the previous exporter.

  • Tooltip titles. Node and edge title fields go through html.escape in Python before serialization (html.py 782, 811, 830). The force-graph HTML tooltip receives &lt;img ...&gt;. The newline-to-<br> step only touches the already-escaped string.
  • JSON in the script tag. All four embedded arrays go through _js_safe, which rewrites </ to <\/ (line 847). The generated file contains exactly two </script> sequences, the two real closing tags. The raw <img text survives only inside JSON string literals, and those fields are rendered either on canvas via fillText or through the esc() helper.
  • Legend community label. Legend entries are built from html.escape(sanitize_label(...)) (line 840) before insertion with innerHTML.

The _html_styles behaviour change is intended: the sidebar stylesheet was rewritten for the collapsible panels and the physics controls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant