Skip to content

fix(spec): preserve manifest extra files - #848

Open
jianguotian wants to merge 1 commit into
apache:mainfrom
jianguotian:codex/manifest-extra-files
Open

jianguotian wants to merge 1 commit into
apache:mainfrom
jianguotian:codex/manifest-extra-files

Conversation

@jianguotian

Copy link
Copy Markdown

Purpose

Recent Paimon Java manifests can own auxiliary files through the nullable _EXTRA_FILES field. Rust must preserve that field when reading and rewriting manifest metadata, otherwise a cross-language rewrite can lose lifecycle ownership of manifest sidecars.

Changes

  • Add nullable extra_files to ManifestFileMeta using the Java field name and schema position.
  • Preserve the distinction between a legacy missing/null value and an explicitly empty list.
  • Include manifest-owned auxiliary files in referenced-file collection and file-size accounting.
  • Add Avro round-trip, legacy compatibility, and referenced-file lifecycle tests.

Verification

  • Targeted manifest metadata, manifest-list, and referenced-file tests
  • cargo clippy -p paimon --lib -- -D warnings

Related Java work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant