[wgtmac]

Summary

Parquet LIST columns that are effectively fixed-length currently pay the full Dremel reconstruction cost on every read. This change adds a page-level fixed-length detector and a direct child-value path for vector-shaped LIST data.

The reader validates all-present definition levels and the periodic repetition-level pattern, derives the list width, then reads values directly across pages and constructs offsets with a fixed stride. The detector handles V1/V2 page boundaries, including pages that continue a list from the previous page.

The optimization is controlled by ArrowReaderProperties::set_fixed_list_fast_path() and is disabled by default. If enabled for a variable-width LIST and a later page changes width after values have been consumed, the reader returns an explicit error instructing callers to disable the property.

Validation

• make -C cpp/build-release -j8 parquet
• make -C cpp/build-release -j8 parquet_shared
• make -C cpp/build-release -j8 parquet-benchmarks
• 158 relevant LIST/FixedSizeList reader tests passed
• Release Google Benchmark built with Conan benchmark/1.9.4

Benchmark

Release, ZSTD, three repetitions, 4 MiB float payload per benchmark case:

• ordinary LIST, PLAIN, width 16: 4.60 ms vs 11.15 ms, 2.42x
• ordinary LIST, BSS, width 768: 0.94 ms vs 5.35 ms, 5.69x
• FixedSizeList, PLAIN, width 768: 5.80 ms vs 8.58 ms, 1.48x
• FixedSizeList, BSS, width 768: 2.55 ms vs 5.29 ms, 2.08x
• ALP remains decode-dominated: ordinary LIST width 768 is 1.34x

The detailed matrix and larger Hardwood-sized stress results are in parquet_vector_reader_optimization.md.

[github-actions]

Thanks for opening a pull request!

This pull request has been automatically converted to a draft because its title doesn't match Arrow's required format.

If this is not a minor PR, could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

After updating the title, you can mark the pull request as ready for review.

See also:

• Other pull requests
• Contribution Guidelines - Contributing Overview