Skip to content

Preserve Planner aggregate semantics through maintenance and serving - #700

Open
zzylol wants to merge 7 commits into
mainfrom
feat/shared-series-quantiles
Open

Preserve Planner aggregate semantics through maintenance and serving#700
zzylol wants to merge 7 commits into
mainfrom
feat/shared-series-quantiles

Conversation

@zzylol

@zzylol zzylol commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Why

Fixes #701 and #702. Reported PromQL workloads need correct window populations, executable aggregates, and whole-expression accuracy contracts.

What

Support shared current-series quantile/TopK/sum/count/average, typed minimum, temporal average, and guarded quantile ratios. Correct both pane and full-window reads while retaining native fallback.

How

Pin Planner PR ProjectASAP/ASAPPlanner#404 at 8a943abc. Planner rules construct typed candidate DAGs; the compiler lowers their operators and compares complete physical cost quotes. Runtime enforces current-value membership, window coverage, and division domains.

Before this PR

Instant aggregates could fail compilation, use temporal populations, or remain native-only. Minimum and temporal average lacked usable lowering. Sliding reads could merge overlapping snapshots or a preceding pane. Quantile division lacked a sufficient expression certificate.

After this PR

Mixed dashboards can maintain temporal summaries alongside shared current-series populations, including replacements, staleness, expiry, and maximum-k reuse. Ten quantiles share compatible state. min_over_time and avg_over_time execute from typed state; checked DDSketch division sizes operands for the requested expression bound and falls back outside its domain.

Evidence

A production-HTTP workload checks 46 queries at two successive evaluations against fresh Prometheus 3.5.0, including 15m/5m windows, multiple cadences, grouped instant aggregates, and both ratio forms. Values, labels, 1% quantile/ratio tolerance, and two zero-denominator fallbacks pass. The current-series process additionally compares 20 query forms through replacements/staleness.

See the developer review. Performance measurements and screenshots: not applicable; correctness quotes are synthetic.

Verification

  • Unit tests: 2,098 backend workspace library tests passed. Focused pane regression also passed after the final range filter.
  • End-to-end tests: 14 compatibility process tests passed; one existing Collector-schema test remains ignored. Includes real Prometheus comparison and durable-summary restart.
  • Other checks: workspace/all-targets clippy with warnings denied, affected data-plane checks after final edits, formatting, and diff checks passed.

Regressions cover typed minimum/average/ratios, equal-valued series counts, shared current membership, invalid division domains, pre-epoch window admission, full-window pending/empty identity, and exclusion of preceding panes.

Architectural decisions

Semantic rewrites stay in Planner. Current-series state retains exact live values; it is not an append-only DDSketch. A 1% ratio requires component accuracy below approximately 0.4975%, not merely a shared 1% sketch. Deployment still requires complete cost evidence.

Limitations and follow-up

Depends on Planner PR 404. The boundary-aligned native oracle requires Prometheus 3.x semantics; 2.x includes the left boundary. Unsupported domains/coverage retain native fallback. Collector schema compatibility and measured performance benefits remain separate work.

Human review — do not complete with an agent

  • The MVP boundary is correct.
  • New conceptual layers or public interfaces are necessary.
  • The before/after description matches the intended product behavior.
  • Human reviewer:
  • Decision and rationale:

@zzylol zzylol changed the title Support shared current-series quantiles and TopK maintenance Preserve Planner aggregate semantics through maintenance and serving Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Physical plans for simple repeated PromQL workloads: min_over_time fails to compile; windows ignore query cadence

1 participant