[python] Cover native dynamic PK events and compacted first-row scans - #9854
Merged
JingsongLi merged 1 commit intoSep 16, 2026
Merged
Conversation
leaves12138
approved these changes
Sep 15, 2026
leaves12138
left a comment
Contributor
There was a problem hiding this comment.
Reviewed 6f0dc0c4019686527e2eae7dfe532b24e10aa76a against bbae07d37545cf2cf65b397007fa68c387bb521f, using Java as the semantic baseline. No blocking findings; approving.
- Confirmed that the only production-file change is a comment:
table_scan.pyhas an identical AST to the base revision. This PR does not change runtime behavior or public APIs. - The incremental migration expectations match Java
DeleteExistingProcessor: emit DELETE in the old partition using the incoming non-partition values. The new assertions check the physical row kind as well as old/new partition predicates, rather than relying on batch results that can hide an incorrect retraction kind. - Checked the synthetic compacted first-row fixtures against Java
CoreOptions.batchScanSkipLevel0andMergeTreeSplitGenerator: overlapping key ranges with disjoint actual keys at L1/L2, fixed/dynamic buckets, skipped pending L0 records, value predicates, and historical snapshots. These test the scan/read paths, not the compactor itself. The new native cases explicitly fail on Python-planner fallback. - Confirmed that the Rust #845 dependency is merged. Rebuilt the Python binding from current Rust main
66f655b30a3861a31b5a0abafd25a8633574049cfor validation.
Local validation on Python 3.10:
- Changed test modules: 12 passed.
- All seven native-plan modules plus split-serializer and global-index tests: 196 passed, 276 native plans exercised.
- Flake8 with the repository's
dev/cfg.ini: passed for all three changed files. - Negative control with the pre-#845 Rust binding: the new cross-partition assertion fails specifically because
(1, 'a', 'moved', 1)is emitted instead of(1, 'a', 'moved', 3). This confirms that the regression test detects the original bug.
GitHub CI is not fully green at review time: several Python jobs, including Rust Plan, are cancelled, the aggregate CI result is failed, and another test-selection check is queued. The local results above do not claim full CI-matrix coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Strengthen native-plan coverage for dynamic primary-key and first-row tables after #9846.
Depends on apache/paimon-rust#845. Keep this PR in draft until that change reaches Rust main: the native CI job intentionally continues to build Rust main, and the new migration assertion detects its current row-kind bug.
Tests
Built the Python binding from paimon-rust commit
c48317e6and ran all seven native-plan test modules,split_serializer_test.pyandglobal_index_test.py: 196 passed, 276 native plans exercised.Flake8 passed for all changed Python files. The migration regression fails against the previous Rust main, as expected.