refactor: split the MinMax aggregation family into Min and Max - #716
Merged
Conversation
A summary's family said `MinMax` and its direction rode alongside it as
a string. `build_backend_aggregation_json` emitted `aggregationType:
MinMax` and recovered "max" from the family in an `if matches!` wedged
inside the `json!` literal; `accumulator_factory` re-derived the
direction with `aggregation_sub_type.eq_ignore_ascii_case("max")`;
`MinMaxAccumulator` carried `sub_type: String` and panicked on anything
that was not "min" or "max". Every one of those is a place where the
direction can disagree with the state it labels, and the failure is
silent: a minimum answered as a maximum.
Worse, `capability_for` mapped both `AggIntent::Min` and
`AggIntent::Max` onto `ExactAgg(MinMax)`, so a deployed minimum summary
was a legal candidate for a `max_over_time` read.
Direction is the family now, end to end:
* `AggregationType::{Min, Max, MultipleMin, MultipleMax}` replace
`MinMax`/`MultipleMinMax`. `FromStr` rejects the retired names rather
than guessing a direction -- there is no safe guess.
* `ExactStateKind::{Min, Max}` and `ExactReadout::Min` give the state
contract and the readout contract the same two-sided shape.
* The wire is `aggregationType: Min` / `Max` with an empty
`aggregationSubType`; nothing reads that field for a direction.
* `MinAccumulator`/`MaxAccumulator` and `MultipleMinAccumulator`/
`MultipleMaxAccumulator` replace the `sub_type`-carrying pair. Each
answers exactly one `Statistic`, refuses to merge with the other
direction, and serializes without a direction byte.
* `RollupReduction::Min` joins `Max`, so `min_over_time` gets the same
derived-rollup fast path `max_over_time` already had; which rollup a
pane feeds follows from the accumulator's type.
This needs ASAPPlanner's own split (`ExactKind::MinMax` -> `Max`
alongside the existing `Min`), so the planner pin moves to
029ff2fe. Two other changes ride in with that pin:
* `BinaryOperator` gained `checked_relative_division` /
`checked_finite_division`, and `avg_over_time` now realizes as a
guarded Sum/Count division instead of staying archive-only --
`avg_over_time_is_not_yet_realizable_matching_capability_for_today`
asserted the old outcome and is rewritten to assert the new one.
* PromQL `count(v)` is a row count upstream now, and the distinct-count
idiom is `count(distinct_over_time(v[w]))`. The four HLL tests that
spelled it `count(v)` are retargeted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
Does this take care of min_over_time compilation from #701 ? |
Contributor
Author
|
min_over_time was taken care by a previous pr |
Before: Planner division guards were discarded when lowering average rewrites and relative divisions into installed binary nodes. After: use a verified original exact subtree where available, or request exact fallback. Remove query placements pruned by exact subtree execution. Regression reproduced before the fix. Validation: 787 control-plane, 1200 data-plane, and 107 shared-type unit tests passed; formatting and clippy for both planes with all targets and warnings denied passed.
Keep the typed minimum and checked division runtime from main, including both compiler paths. Preserve full-window rollup admission and port the min/max wire split to the renamed backend emitter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A summary's family said
MinMaxand its direction rode alongside it as a string.build_backend_aggregation_jsonemittedaggregationType: MinMaxand recovered"max"from anif matches!wedged inside thejson!literal.accumulator_factoryre-derived the direction withconfig.aggregation_sub_type.eq_ignore_ascii_case("max").MinMaxAccumulatorcarriedsub_type: Stringand panicked on anything that was not"min"or"max"; the direction also occupied a ninth byte in its serialized form.catalog_resolver::supportscross-checkedaggregation_sub_typeagainst the family before admitting a candidate.Every one of those is a place where the direction can disagree with the state it labels, and the failure is silent — a minimum answered as a maximum, not an error.
Worse,
capability_formapped bothAggIntent::MinandAggIntent::MaxontoExactAgg(MinMax), so a deployed minimum summary was a legal candidate for amax_over_timeread.What
Direction is the family now, end to end.
asap_types::AggregationTypeMinMax,MultipleMinMaxMin,Max,MultipleMin,MultipleMaxasap_types::ExactStateKindMinMaxMin,Maxasap_types::ExactReadoutMaxonlyMin,MaxaggregationType: MinMax+aggregationSubType: maxaggregationType: Min/Max, sub-type emptyMinMaxAccumulator { sub_type: String }MinAccumulator/MaxAccumulatorMultipleMinMaxAccumulator { sub_type: String }MultipleMinAccumulator/MultipleMaxAccumulatorRollupReduction::MaxRollupReduction::{Min, Max}AggregationType::from_strrejects the retired names (MinMax,MultipleMinMax,min_max, …) rather than guessing a direction. There is no safe guess, and a wrong one is silently wrong.Statistic, refuses to merge with the opposite direction, and serializes without a direction byte.min_over_timegets the derived-rollup fast pathmax_over_timealready had. Which rollup a pane feeds follows from the accumulator's type instead of from asub_typestring that had to agree with it.capability_formapsAggIntent::Min→ExactAgg(Min)andAggIntent::Max→ExactAgg(Max); a new test pins that neither satisfies the other.Planner pin
Needs ASAPPlanner's own split (
ExactKind::MinMax→Maxalongside the existingMin), so the four ASAPPlanner deps moveca7546d→029ff2f— ProjectASAP/ASAPPlanner#407.Two unrelated upstream changes ride in with that pin and are handled here:
BinaryOperatorgainedchecked_relative_division/checked_finite_division, andavg_over_timenow realizes as a guarded Sum/Count division instead of staying archive-only.avg_over_time_is_not_yet_realizable_matching_capability_for_todayasserted the old outcome; it is rewritten asavg_over_time_realizes_as_a_checked_sum_over_count_division.count(v)is a row count upstream now (correct PromQL), and the distinct-count idiom iscount(distinct_over_time(v[w])). The four HLL tests that spelled itcount(v)are retargeted. Restoring a by-less cross-source merged cardinality is the second commit of feat(control_plane): adopt asap_plan::bind::implement_tree (Step A) #407.Deliberately not in scope
summary_family_matches_exactstill excludesMin/Maxfrom the legacy family-discovery path. It excluded them because direction was unrecoverable once a summary reachedAggKind::ExactAgg— a reason this PR removes — but admitting them widens candidate discovery beyond the family split. The doc comment now records that, and the test is renamed to say it is the discovery path, not the ambiguity, that keeps them out.Compatibility
PolicyFingerprinthashesaggregation_typeandaggregation_sub_type, so extremum summaries content-address differently after this change. Both sides of that wire move together in this PR; persisted streaming-configs and on-disk parts written asMinMaxAccumulatorby an older build are not readable by the new byte factory.Testing
cargo test --workspace -- --test-threads=1in CI's configuration (ASAP_TEST_TIMEOUT_SCALE=4,ASAP_E2E_CONTROL_PLANE_BINset), toolchain 1.98.0.cargo clippy --workspace --all-targetsandcargo fmt --checkclean.Everything green except six
asapquery_compatibility_process_e2etests. Those six fail identically on unmodifiedorigin/main(b1a58ca, pin ca7546d) in the same configuration — verified against a pristine baseline worktree, not assumed:They are pre-existing on
mainin this environment and are not touched by this change.🤖 Generated with Claude Code