[python] Expand native planning and align incremental reads with Java

[JingsongLi]

Purpose

Expand native scan coverage while keeping Java's read semantics. This follows #9839 and uses the split streaming flag and selector normalization merged in apache/paimon-rust#841. The Rust-main CI job checks the new API so incremental coverage cannot silently skip on an older runtime.

• Enable scan.version, including tag precedence and historical schemas; ordinary postpone-bucket batch reads; and supplied scored DE index results with scores attached to the ranges retained by Rust.
• Refine native PK batch splits through the existing Python sorted-index reader, using the planned snapshot and preserving merge-required splits. Index evaluation is still performed in Python.
• Align timestamp incremental reads with Java: retain physical versions and row kinds across APPEND commits, include L0, and omit endpoint DVs and automatic global indexes. Explicit split DVs still apply. Preserve the streaming bit in the split codec and wrappers; continuous Python delta/changelog reads also retain retracts, while their initial snapshot remains merged.

For example, an incremental window containing two committed values for the same key now returns both events instead of only the final value. Both planners reject manifest DELETE entries in APPEND deltas before file reconciliation.

Chunk shuffle, query authorization, supplied PK index results, continuous-stream planning and write planning retain their existing Python entrypoints. Native planning remains opt-in.

Tests

• Built a wheel from paimon-rust main 277213ad and ran 150 planner, codec, global-index and incremental-read tests: all passed, with 46 native plans exercised.
• Real committed-file tests run against both Python planning and a local Rust wheel; native cases reject fallback.
• Cover postpone buckets/shards/limits, scored DE selections and DVs, historical schemas, sorted-index snapshot selection and uncovered files, same-key versions, four row kinds, first-row, malformed manifests and explicit split DVs.
• Relevant planner, codec, time-travel, streaming and sorted-index suites: 282 passed. Two unrelated modules were skipped because optional PIL/lumina_data dependencies are absent.
• Flake8 and git diff --check passed.

[leaves12138]

Reviewed 32c650c with Java as the compatibility baseline. No blocking findings.

Checked timestamp incremental and continuous delta/changelog reads, preservation of physical PK events and all four row kinds, L0 visibility, omission of endpoint deletion vectors and automatic indexes, explicit split deletion vectors, and streaming-flag propagation. Also checked scan.version precedence and historical schemas, postpone-bucket visibility, scored DE selection, and snapshot-specific PK sorted-index refinement without pruning merge-required splits.

Local validation on Python 3.10:

• 287 related planner, reader, codec, time-travel, streaming and sorted-index tests passed.
• 45 additional Java interoperability/oracle tests passed, including Java-written data and serialized streaming splits; native cases explicitly disallow fallback.
• Flake8 passed for all 21 changed Python files.

The native checks use a freshly rebuilt wheel from the merged paimon-rust main commit 277213ad23c0c42394d9f47a8b355baeaf54ddcb, including the final #841 serialization API. GitHub's Python CI jobs are still queued; the full CI matrix has not been verified by these local checks.