[JingsongLi]

Purpose

Maintain declared global indexes synchronously after append/Data Evolution writes when write-only=false, so callers do not need a separate index build after each write.

Brief change log

• Add Java-style per-family column declarations (global-index.<family>.index-columns) and per-column native options. Reuse the existing scalar, full-text and vector builders.
• Capture the current schema and a source snapshot, fill missing row-ID coverage, coalesce partial shards, and atomically publish additions with retired references. Vector algorithm or metric changes replace incompatible payloads across the column; historical files remain readable.
• Keep replacement selection and retirement in auto_index. Shared planning captures and filters one immutable index view; concrete builders use that view for both coverage calculation and overlap validation.
• Replan after concurrent snapshot/schema changes. Use a stable identity derived from the data commit for idempotent recovery, and retain submitted files when publication is uncertain. Recovery also resumes an interrupted index phase after compaction has completed.
• Share field configuration between automatic/manual builds, planned vector shards, and FULL/DETAIL raw fallback, preserving explicit option precedence.
• Maintain declared indexes after manual compaction as well.

Data, compaction and index publication are separate transactions because global row IDs must be assigned before indexing. An index failure leaves the data committed; retrying with the same commit user and identifier resumes maintenance.

Tests

• cargo test --locked -p paimon --all-targets --features vortex — passed; core unit tests: 4728 passed, 6 ignored.

• cargo +1.98.0 clippy --locked --all-targets --workspace --features vortex -- -D warnings — passed.

• cargo +1.98.0 fmt --all -- --check — passed.

• Regression coverage includes CAS conflicts, uncertain publication, concurrent recovery, append/overwrite/manual compaction recovery, stale REST committers, FULL/DETAIL fallback with no or partial coverage, manual builder defaults/overrides, and vector algorithm/metric changes with interrupted publication and replay.

• Filtered-view regression tests cover BTree, full-text and vector builders: captured snapshots stay fixed, only the selected gap is rebuilt, unrelated coverage and overlap guards remain intact, and building does not alter published references or historical files.

API and Format

New table options declare automatically maintained indexes. Maintenance runs with write-only=false (the default) only when indexes are declared; write-only=true skips automatic maintenance. Supported tables have row tracking, no primary key and bucket=-1.

Existing global-index builders, payload formats and index manifests are reused. Primary-key index maintenance is outside this change.

Documentation

Update the compaction and SQL guides with declarations, option precedence, definition replacement, and recovery APIs.

[leaves12138]

LGTM