Skip to content

fix(schema): list persisted schemas independently of catalog cache - #1260

Merged
JingsongLi merged 1 commit into
apache:mainfrom
XiaoHongbo-Hope:codex/persisted-schema-listing
Oct 31, 2026
Merged

JingsongLi merged 1 commit into
apache:mainfrom
XiaoHongbo-Hope:codex/persisted-schema-listing

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

After #1259 seeds the schema cache from the REST catalog response, $schemas.update_time can report the time the client reconstructed the schema instead of its persisted timestamp. Python/native parity tests fail even on empty tables.

Read enumerated schema files through a fresh file-backed manager in list_all. Keep catalog-resolved schemas available to ordinary schema lookups, including catalogs without schema files. Add a regression with different persisted and catalog-resolved timestamps.

Tests

  • Reproduced the three $schemas parity failures on Apache Paimon master 1418eb504e with Rust 74834711, including empty tables and branch reads.
  • Added a regression proving listing uses the persisted timestamp while ordinary schema lookup retains the catalog cache.
  • cargo test --release -p paimon --lib table::schema_manager::tests: 8 passed, including the new regression.
  • cargo fmt --all -- --check and git diff --check passed.
  • The full Python/native suite has not yet been rerun against a rebuilt extension containing this fix.

Companion Python fixture fixes: apache/paimon#10900. Those fix the other 15 Native CI failures without changing schema parity assertions.

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit 8e765e7 into apache:main Oct 31, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants