[python] Support snapshot properties in table commits - #9692
Merged
JingsongLi merged 1 commit intoSep 8, 2026
Conversation
Allow batch and stream commits to atomically attach application state to the generated Paimon snapshot, matching the capability available through Java ManifestCommittable. Co-Authored-By: Codex <noreply@anthropic.com> AI-Model: gpt-5 Co-Authored-By: Codex <noreply@ai-tool.com> Co-Authored-By: Codex <noreply@openai.com> AI-Contributed/Feature: 76/76 AI-Contributed/UT: 101/101
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PyPaimon can deserialize
Snapshot.properties, but its table commit API has no way to populate them. Java supports the same capability throughManifestCommittable.addProperty. This PR closes that gap so Python applications can atomically publish recovery or checkpoint state with the data represented by the snapshot, instead of maintaining an inconsistent sidecar.The direct LeRobot capture writer in #9665 is the first consumer: it uses the properties to resume global frame, episode, and task indices without scanning the table.
Changes
snapshot_propertiesmapping to batch and stream table commits.Testing
python -m pytest -q pypaimon/tests/table_commit_test.py pypaimon/tests/table/simple_table_test.py(28 passed).git diff --checkpass for all changed files.Notes
An empty append still does not create a snapshot solely for properties. This PR adds properties to snapshots produced by data commits; it does not add metadata-only commits.
Related: #9665