Skip to content

Find moves in make_patch without rewriting all later operations - #208

Merged
stefankoegl merged 3 commits into
masterfrom
claude/inspiring-ride-njgoou
Oct 10, 2026
Merged

stefankoegl merged 3 commits into
masterfrom
claude/inspiring-ride-njgoou

Conversation

@stefankoegl

Copy link
Copy Markdown
Owner

make_patch took about 150 s to diff list(range(10000)) against its
reverse, and 10 s for a random permutation of 3000 items. Profiling
showed the time in DiffBuilder._adjust_following: whenever a 'remove'
and a later 'add' (or the other way round) became a 'move', all
operations after the earlier one were walked and their array indices
rewritten, which is quadratic in the number of operations.

Operations now refer to array items by _ArrayItem objects instead of
indices. These do not change when a move replaces an earlier operation,
so nothing has to be rewritten. Each array keeps all items it has during
the diff in one order, by integer labels, and the items it has at the
moment; the indices that the moves need are found with bisect. When the
diff is done, the operations are replayed once to get their indices.

The order puts an inserted item directly after the item before it, so
before removed items. This is how _adjust_following placed them, and
the patches are the same as before: they were compared with the
previous version on about 170000 random and Hypothesis generated pairs
of documents.

take_index scanned all stored unhashable values (lists and objects) for
one equal to the given one. They are now grouped by a hashable key that
equal values share, and only values in the same group are compared.

The two cases above now take 0.45 s and 0.08 s.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01RE4gSqjC7bhmzDr3325NKC

make_patch took about 150 s to diff list(range(10000)) against its
reverse, and 10 s for a random permutation of 3000 items. Profiling
showed the time in DiffBuilder._adjust_following: whenever a 'remove'
and a later 'add' (or the other way round) became a 'move', all
operations after the earlier one were walked and their array indices
rewritten, which is quadratic in the number of operations.

Operations now refer to array items by _ArrayItem objects instead of
indices. These do not change when a move replaces an earlier operation,
so nothing has to be rewritten. Each array keeps all items it has during
the diff in one order, by integer labels, and the items it has at the
moment; the indices that the moves need are found with bisect. When the
diff is done, the operations are replayed once to get their indices.

The order puts an inserted item directly after the item before it, so
before removed items. This is how _adjust_following placed them, and
the patches are the same as before: they were compared with the
previous version on about 170000 random and Hypothesis generated pairs
of documents.

take_index scanned all stored unhashable values (lists and objects) for
one equal to the given one. They are now grouped by a hashable key that
equal values share, and only values in the same group are compared.

The two cases above now take 0.45 s and 0.08 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RE4gSqjC7bhmzDr3325NKC

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The core diff and move-indexing algorithm is substantially rewritten and warrants final human validation despite strong regression coverage.

0 open findings

What changed in this PR

Optimizes make_patch by replacing repeated index rewrites with stable array-item references and replaying operations once.

Changes:

  • Tracks array items with ordered labels and resolves indices during replay.
  • Groups unhashable values by structural keys for faster move matching.
  • Adds correctness and performance regression tests for large and nested moves.
File Description
jsonpatch.py Implements stable array locations, operation replay, and grouped unhashable lookup.
tests.py Covers index-sensitive, nested, large-list, and unhashable-value moves.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

claude added 2 commits October 9, 2026 21:55
Conflicts in jsonpatch.py:
- store_index/take_index: master's _move_key, which keys moves by the
  JSON serialization of values, already avoids scanning unhashable
  values one by one, so it replaces the _hashable grouping of this
  branch, which is dropped along with index_storage2.
- Helpers: master's _positional_cost and _sorted_members are kept next
  to the _ArrayItem classes of this branch.

test_moves_change_indices_in_nested_list gets an input for which the
list alignment from master still pairs up the list items by position,
so a move in the outer array still shifts operations in the nested one.
Its expected patch is the one master gives. The duplicate import of
random in tests.py is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RE4gSqjC7bhmzDr3325NKC
Conflict in tests.py: both sides added tests to OptimizationTests at
the same place; all of them are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RE4gSqjC7bhmzDr3325NKC
@stefankoegl
stefankoegl merged commit 1b30499 into master Oct 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants