Skip to content

fix: sum projected column byte sizes in project_statistics - #27052

Open
joseph-isaacs wants to merge 1 commit into
apache:mainfrom
joseph-isaacs:ji/project-stats-byte-size
Open

joseph-isaacs wants to merge 1 commit into
apache:mainfrom
joseph-isaacs:ji/project-stats-byte-size

Conversation

@joseph-isaacs

Copy link
Copy Markdown
Contributor

Closes #27050

In ProjectionExprs::project_statistics computing the Statistics::total_byte_size the ColumnStatistics::byte_size is ignored even when present.

This PR sums the columns statastics if present and fallbacks to the ProjectionExprs::calculate_total_byte_size is absent

ProjectionExprs::project_statistics kept the input's total_byte_size,
marked inexact, whenever a projected column is variable width, even when
every projected column has a known byte_size. Sources that report column
byte sizes then show every projected scan as large as the whole table, so
join selection picks the wrong build side.

Sum the projected columns' byte sizes when they are all known, as
Statistics::with_fetch already does, and keep the old fallback otherwise.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs
joseph-isaacs force-pushed the ji/project-stats-byte-size branch from 23f8f6c to f655c82 Compare November 3, 2026 14:56
@github-actions github-actions Bot added the physical-expr Changes to the physical-expr crates label Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-expr Changes to the physical-expr crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ProjectionExprs::project_statistics computes approximate Statistics::total_byte_size ignoring column stats

1 participant