feat(parquet): support page-level (page index) filtering for nested column types (patch arrow)

[zhf999]

Purpose

Page-level filtering (row skipping driven by the Parquet page index / OffsetIndex)
was previously disabled whenever the read schema contained a nested column
(struct / list / map). ParquetFileBatchReader::SetReadSchema detected any nested
field and fell back to the non-page-filtered path, so nested reads could not skip
non-matching pages at the I/O and decode level.

This change enables page-level filtering for nested column types and reworks the
reader so that each leaf column is filtered and driven independently.

What changed:

• Enable page filtering for nested schemas. Removed the has_nested_field
  workaround in ParquetFileBatchReader that skipped page-index pushdown for both
  the predicate and the selection-bitmap paths when any nested field was present.

• Nested-aware page-filtered reader (PageFilteredRowGroupReader). Each
  top-level field is read through an Arrow ColumnReader tree
  (FileReader::GetColumn + leaf projection). A per-leaf data_page_filter is
  installed to skip non-matching pages during decode, and each leaf is then driven
  independently with reset / skip / read primitives. The result schema is built
  from the actually-produced column types (needed because column pruning can change
  nested types).

• Arrow patch (cmake_modules/arrow.diff). Instead of a single monolithic
  entry point, parquet::arrow::ColumnReader gains small, per-leaf primitives:
  LeafColumnIndices(), ResetLeaf(), SkipRecords() and ReadRecords()
  (returning the actual record count), implemented across the leaf / extension /
  list / struct readers. parquet::arrow::FileColumnIterator::NextChunk() is made
  virtual so the page-filter wiring lives in a Paimon-side
  PageFilteringColumnIterator subclass, keeping the patch surface minimal.

Tests

Unit tests in src/paimon/format/parquet/page_filtered_row_group_reader_test.cpp:

• Nested row-group and page filtering: NestedStructColumnRowGroupFilter,
  NestedListColumnRowGroupFilter, NestedMapColumnRowGroupFilter,
  MultipleAdjacentNestedColumns, MultipleNestedColumns.
• Leaf / sub-column projection: NestedStructColumnOnlyReadIdField,
  NestedStructSubColumnProjection.
• Misaligned pages across nested leaves: NestedColumnsMisalignedPages — writes a
  struct<key:int64, s:struct<x:int32,y:int64>, tags:list<int32>, props:map<utf8,int32>> with write_batch_size=1 and a byte-based data page size
  so leaves paginate at different (and, for the utf8 map key, irregular) row counts;
  selects two disjoint row ranges via a bitmap and deep-compares every returned row
  against the source data to catch any per-leaf skip/read desync.
• Existing flat page-filtering, ComputePageRanges, streaming and bitmap cases
  continue to pass.

API and Format

• Paimon public API under include/paimon/ is unchanged.
• The bundled Arrow build is patched via cmake_modules/arrow.diff: new virtual
  methods on parquet::arrow::ColumnReader and a virtual
  FileColumnIterator::NextChunk(). This affects only the vendored Arrow used by
  the read path, not any symbol shipped from include/paimon/.
• No change to the Paimon storage format, on-disk layout, or protocol; this is a
  read-path optimization.

Documentation

No user-facing documentation change; this is an internal read-path optimization.

Generative AI tooling

Generated-by: GLM 5.2

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.