Repository navigation
Conversation
### What problem does this PR solve? Issue Number: None Related PR: apache#69776 Problem Summary: Segment readers keep row ranges computed before a TopN runtime bound arrives or tightens. Scan-key normalization can also remove a shared runtime predicate on a leading storage key. Readers then miss later bound updates. Publish versioned predicate snapshots and use page Zone Maps to refresh unread row ranges between batches. Apply the first bound immediately and throttle later refreshes by the number of raw rows read. Preserve scan progress, bitmap holes, and read direction. Refresh the bitmap cursor, cached cardinality, and filter counters together. Keep shared runtime predicates during static scan-key normalization while still removing static predicates covered by scan keys. In the existing ascending TopN check with 1,048,576 rows and three sort keys, ScanRows fell from 1,048,576 to 16,384 and TotalPagesNum fell from 372 to 6. These figures compare filtering disabled and enabled on the same patched build. The query results were identical. ### Release note TopN scans can skip unread pages when runtime bounds arrive or tighten. Runtime predicates remain active after scan-key normalization. ### Check List (For Author) - Test: Manual checks of commit boundaries, dependencies, generated-result preservation, whitespace, clang-format v16, and header hygiene. Include BE unit tests and two TopN regression suites. Builds and tests were not rerun for this commit split. - Behavior changed: Yes. TopN scans can read fewer rows and pages. This commit retains inclusive runtime bounds. - Does this need documentation: No
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Local pipeline review — ✅ PASSschema: doris-repo-review/v1
status: PASS
pr: apache/doris#70020
commit: 1b8d5fd63c7547c2964ccf45cf6ce32a5540b659
base: 156dbe71dc63ed6e919c131d33279264b73187f6
reviewed_at: 2026-11-03T12:12+08:00
reviewer: jacktengg
model: gpt-6-astra
effort: xhigh
findings: {blocker: 0, major: 0, minor: 0, nit: 0}
rounds: 1
converged: trueNotes for maintainers
Reviewed locally with the |
|
run buildall |
TPC-H: Total hot run time: 28745 ms |
TPC-DS: Total hot run time: 151820 ms |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
ClickBench: Total hot run time: 24.07 s |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
HappenLee
left a comment
There was a problem hiding this comment.
Reviewed the versioned predicate snapshots, unread-range intersection, and bitmap cursor/cardinality updates. Follow up with base/head benchmarks for frequently tightening bounds with low page-pruning selectivity.
### What problem does this PR solve? Issue Number: None Related PR: apache#70020 Problem Summary: The SharedPredicate constructor taking another SharedPredicate and a column ID has no callers in production code or tests. RuntimePredicate creates instances with a column ID and name, while clone() returns the existing shared instance. Remove the unused overload without changing runtime behavior. ### Release note None ### Check List (For Author) - Test: Manual test: searched all tracked references and verified the factory construction and clone paths; clang-format 16 and check-format passed for the modified header; BE build-hygiene and git diff --check passed. No build or unit tests run for this unused-code deletion. clang-tidy was attempted but could not complete because the existing analysis environment cannot locate stddef.h. - Behavior changed: No - Does this need documentation: No
|
run buildall |
Local pipeline review — ✅ PASSschema: doris-repo-review/v1
status: PASS
pr: apache/doris#70020
commit: 0e20c230d359fab1102ca1491d004643f72ecfb9
base: 156dbe71dc63ed6e919c131d33279264b73187f6
reviewed_at: 2026-11-03T21:51+08:00
reviewer: jacktengg
model: gpt-6-astra
effort: xhigh
findings: {blocker: 0, major: 0, minor: 0, nit: 0}
rounds: 1
converged: trueNotes for maintainers
Reviewed locally with the |
|
run buildall |
TPC-H: Total hot run time: 28651 ms |
TPC-DS: Total hot run time: 151211 ms |
ClickBench: Total hot run time: 23.97 s |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
What problem does this PR solve?
Issue Number: None
Problem Summary:
Segment readers keep row ranges computed before a TopN runtime bound arrives or tightens. Scan-key normalization can also remove a shared runtime predicate on a leading storage key. Readers then miss later bound updates.
Publish versioned predicate snapshots and use page Zone Maps to refresh unread row ranges between batches. Apply the first bound immediately and throttle later refreshes by the number of raw rows read. Preserve scan progress, bitmap holes, and read direction. Refresh the bitmap cursor, cached cardinality, and filter counters together. Keep shared runtime predicates during static scan-key normalization while still removing static predicates covered by scan keys.
In the existing ascending TopN check with 1,048,576 rows and three sort keys, ScanRows fell from 1,048,576 to 16,384 and TotalPagesNum fell from 372 to 6. These figures compare filtering disabled and enabled on the same patched build. The query results were identical.
Release note
TopN scans can skip unread pages when runtime bounds arrive or tighten. Runtime predicates remain active after scan-key normalization.
Check List (For Author)
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)