Skip to content

feat: a graph can be pickled, and copy() works for graphs of any size - #44

Merged
petercorke merged 2 commits into
mainfrom
feat/graph-pickle
Oct 11, 2026
Merged

petercorke merged 2 commits into
mainfrom
feat/graph-pickle

Conversation

@petercorke

Copy link
Copy Markdown
Owner

Summary

A graph can be pickled, and copy() / copy.deepcopy work for graphs of any size.

What was wrong

  • pickle.dumps(g) failed for every graph with a built-in metric: L1/L2/SE2 were local functions inside _metricfunc, which pickle cannot find by name.
  • g.copy() (which is copy.deepcopy) raised RecursionError for a graph with a path of about 100 vertices, and pickle did the same at about 2000. A graph is a web of mutual references (vertex, edge, vertex, ...) that pickle and deepcopy follow recursively, so the recursion depth grows with the longest chain, not the size of the graph.

Change

  • The graph describes itself flatly: a list of vertices, then a list of edges that refer to their end vertices by index. They are replayed through the vertices' own _attach hook in order of addition, so every vertex's edge lists (and a DGraph's in-lists) come back in their original order, including parallel edges and self-loops.
  • Vertices and edges are rebuilt without calling __init__, so user subclasses with their own constructor signature (e.g. MVTB ViewPoint(x, fixed, color)) restore correctly, with their extra attributes.
  • The built-in metrics are module level and saved by name; a user function pickles if importable at module level. A lambda can be copied but not pickled (documented).
  • The state carries a version number, so the layout can evolve; a pickle is not an archival format (export() is the portable one).

Behaviour to review

  • A vertex or an edge on its own refuses to be pickled or copied (TypeError: ... pickle or copy its graph instead): it has no meaning apart from its graph. Consequences: copy.copy(e) / copy.deepcopy(e) on an edge now raise (an existing test that asserted otherwise was updated), and vertex/edge data that refers to another vertex or edge cannot be pickled or copied.
  • copy.copy(g) goes through the same hook, so it is a structural copy (new vertices and edges sharing coordinate arrays and data).

Tests

tests/test_pickle.py (37 tests, both graph kinds): round trip of every observable including order, every pickle protocol, copy() independence, settings and metrics, 5,000-vertex chains for pickle / copy() / deepcopy, subclasses with their own constructors, the refusals. Run against the parent they fail (36 of 43), which is the point.

Closes

Supersedes the copy half of #36 together with #43 (already merged).

🤖 Generated with Claude Code

Pickle and deepcopy followed the web of vertex and edge references
recursively, so copy() overflowed the stack for a path of about a hundred
vertices, and a graph with a built-in metric could not be pickled at all
because the metric was a local function.

The graph now describes itself flatly: vertices, then edges that refer to
their end vertices by index, replayed in order of addition so that every
vertex's edge lists are rebuilt in their original order.  Built-in metrics
are module level and saved by name.  A vertex or edge on its own refuses to
be pickled or copied, since it has no meaning apart from its graph.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@codacy-production

codacy-production Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 57 complexity · 0 duplication

Metric Results
Complexity 57
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@codecov

codecov Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.21429% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 94.79%. Comparing base (b63389f) to head (9059faa).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/pgraph/PGraph.py 98.21% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #44      +/-   ##
==========================================
+ Coverage   94.61%   94.79%   +0.17%     
==========================================
  Files           3        3              
  Lines        1208     1249      +41     
==========================================
+ Hits         1143     1184      +41     
  Misses         65       65              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The built-in metrics return float(...) so that mypy, against the CI's numpy,
sees a float rather than floating[Any].  The tests pickle their own graphs,
so the pickle calls go through one marked helper pair.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@petercorke
petercorke merged commit 9fe9d61 into main Oct 11, 2026
12 checks passed
@petercorke
petercorke deleted the feat/graph-pickle branch October 11, 2026 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant