Skip to content

fix(sql): a view's HAVING reads every aggregate it names; GREATEST/LEAST skip NULL; DATEDIFF per group and by unit - #409

Merged
fupelaqu merged 1 commit into
mainfrom
fix/view-having-every-aggregate
Oct 2, 2026
Merged

fupelaqu merged 1 commit into
mainfrom
fix/view-having-every-aggregate

Conversation

@fupelaqu

@fupelaqu fupelaqu commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

What

A materialized view's HAVING reads every aggregate it names. Before, a view declared one aggregate per comparison, so on 0.23.0 a COALESCE(MAX(a), MIN(b)) condition was silently dropped. On main it was refused, with remedies that pointed at each other.

  • One core function, Criteria.bucketMetrics / Having.metricNames, lists every aggregate a HAVING condition reads. Core's view rules, the search path and (next) extensions read it.
  • The "two aggregates compared" rule is gone. The catch-all rule now names the metric the view does not create.
  • Two-aggregate comparisons and functions over several aggregates (COALESCE, GREATEST, LEAST, SIGN, …) deploy.
  • A bare ISNULL(MIN(a)) / ISNOTNULL(MIN(a)) deploys again, with exactly the alias spelling's filter, as on 0.23.0.
  • A per-group calculation channel (BucketScriptTransformAggregation, with the script params it reads, e.g. __now__) lets a view compute SELECT arithmetic over aggregates (MAX(a) - MIN(b) AS d) and filter on it. Such a column used to be declared in the view schema and never stored.

Search: HAVING over functions of aggregates

  • ISNULL / ISNOTNULL and the DATEDIFF family parse an aggregate operand as the aggregate; it used to be a bare token. This ends a false "different aggregate" refusal of ISNULL(MIN(r.a)).
  • A SELECT alias inside a HAVING function (COALESCE(max_a, min_b) > 4) is replaced by its aggregate at every depth. Each shape now has one verdict and one message across its spellings, and is accepted again where expressible, as on 0.23.0.
  • GREATEST / LEAST skip NULL arguments in HAVING, as documented and as in WHERE.

DATEDIFF / DATE_DIFF / TIMESTAMPDIFF

  • Over aggregates, they run per group. A date aggregate reaches the calculation as epoch millis and is converted once.
  • HOUR / MINUTE / SECOND compute on timestamps, as elapsed whole units. DAY and above count calendar dates, as before.
  • Literal operands are typed, and QUARTER compiles.
  • A window-function operand with no GROUP BY is refused by name.

Docs: the DATEDIFF sign is date2 - date1, the engine's behaviour, with corrected examples, the units, literals, and MySQL's two-argument DATEDIFF(a, b) = a - b.

Measured (ES 8.18.3 through the gateway; oracles computed independently in Scala)

Check main this branch
HAVING population, 566 statements derived from source (search + views) — 333 unchanged, 233 changed, all in decided classes
Search requests — byte-identical except 12 GREATEST/LEAST renders; 23 refused → accepted; 0 accepted → refused
View filters: aggregates declared = aggregates read — 119/119
Remedies that lead to an accepted statement — 500/500
GREATEST/LEAST over aggregates, 900 statements 437 wrong, 116 errors 0 wrong, 0 errors
DATEDIFF family (spellings × units × operands × 4 venues), 3,066 statements 2,890 errors 0 wrong, 0 errors
DATEDIFF over aggregates, 72 statements errors 0 wrong, 0 errors
Spellings that used to be refused, 233 statements refused 0 wrong, 0 errors
Existing NULL (1,628) and COALESCE (398) populations — still 0 wrong
Newly emitted Painless APIs, allow-lists of 6.8.23 / 7.17.29 / 8.18.3 / 9.0.3 — 29 × 4, 0 missing
Parse / emission cost, 5 interleaved rounds against main — ×0.969 / ×0.978, nothing slower

Suites: sql 1785, es6bridge 256, softclient4es8-sql-bridge 263, core 1148. ES 8.18.3 integration: 9 suites, 327/327. Lint (headerCheck scalafmtSbtCheck scalafmtCheck test:scalafmtCheck) is clean.

Not run locally, left to CI: ES 6.8 / 7.17 / 9.0, the Scala 2.12 cross-compile, the es7 / es9 bridges. Extensions and arrow are not touched by this PR.

Behaviour changes (release notes)

  • Views: more HAVING shapes deploy, and SELECT arithmetic over aggregates is stored. Each needs the extensions follow-up below.
  • Search:
    • qualified ISNULL(MIN(r.a)) and alias-in-function HAVING are accepted;
    • GREATEST/LEAST skip NULLs in HAVING, so groups with an empty aggregate may change;
    • DATEDIFF sub-day units, QUARTER, literal operands and per-group DATEDIFF now work; they were errors;
    • newly refused by name: DATEDIFF over a window function with no GROUP BY (was a parse error), GREATEST(MAX(a), NULL) (was an Elasticsearch compile error), and HAVING nested(s > 5) (views dropped the condition).
  • API:
    • new Criteria.bucketMetrics (public), Having.metricNames, BucketScriptTransformAggregation, SingleSearch.transformBucketScripts / transformBucketScriptOperands;
    • removed Expression.extractAllMetricsPath's override and Identifier.allMetricsPath;
    • downstream projects must rebuild against this core.

⚠️ Sequencing: no extensions release between this core's publish and the extensions PR

Once the "two aggregates compared" rule is gone, extensions main still declares one aggregate per comparison, so such a view would come out partial or empty (HTTP 200) until the extensions PR lands. That PR:

  • declares Having.metricNames in the view filter, and fails loudly instead of dropping a filter;
  • emits the bucket_script channel and binds its params (__now__);
  • fixes a false "produced by more than one aggregate" refusal;
  • adds RUN view tests.

🤖 Generated with Claude Code

…AST skip NULL; DATEDIFF per group and by unit

One core function (Criteria.bucketMetrics / Having.metricNames) lists every
aggregate a HAVING condition reads, for core's view rules, search and
extensions. A view's HAVING may compare two aggregates or apply COALESCE,
GREATEST, LEAST or SIGN over several; a bare ISNULL(MIN(a)) deploys again;
a per-group calculation channel (BucketScriptTransformAggregation) lets a
view store and filter SELECT arithmetic over aggregates.

ISNULL/ISNOTNULL and the DATEDIFF family parse an aggregate operand as the
aggregate; a SELECT alias inside a HAVING function is replaced by its
aggregate at every depth; GREATEST/LEAST skip NULL arguments in HAVING.

DATEDIFF / DATE_DIFF / TIMESTAMPDIFF run per group over aggregates, compute
HOUR/MINUTE/SECOND on timestamps (DAY and above keep calendar dates), type
literal operands and support QUARTER; over a window function with no GROUP
BY they are refused by name. Docs follow the engine's date2 - date1 sign.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@fupelaqu
fupelaqu marked this pull request as ready for review October 2, 2026 17:25
@fupelaqu
fupelaqu merged commit 0e8188c into main Oct 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant