Repository navigation
[C++][Parquet] Default to V2 data pages #55272
Description
Describe the enhancement requested
V2 data pages are very old and probably supported by every implementation post-2020. Given they allow selective compression of data pages and other useful stuff (such as num_nulls), we should probably write them by default.
Component(s)
C++, Parquet
Activity
pitrou commentedon Oct 26, 2026
@wgtmac @HuaHuaY Thoughts?
wgtmac commentedon Oct 27, 2026
I just found https://stackoverflow.com/questions/77654784/what-is-the-difference-between-data-page-version-1-0-and-2-0-in-parquet-files which is answered by @emkornfield :)
We have enabled all supported encodings in our production environment for years but we still default data page to v1 for better compression ratio. I don't know if data page v2 has any potential bug but overall I'm in favor of switching the default to v2.
adamreeve commentedon Oct 28, 2026
Can you provide some more details on this @wgtmac, why would v1 give better compression?
wgtmac commentedon Oct 28, 2026
It was several years ago when we ran TPC-H/TPC-DS/SSB benchmarks. If I remember correctly, v1 data pages have better compression ratio because of better compression on definition levels as they are all flat schemas.
pitrou commentedon Oct 28, 2026
It'd be interesting to know by how much.