Skip to content

[IOTDB-17635] Fix incorrect ordering of multiple window RANK functions#17706

Open
DaZuiZui wants to merge 6 commits into
apache:masterfrom
DaZuiZui:fix-window-function-over-identity
Open

[IOTDB-17635] Fix incorrect ordering of multiple window RANK functions#17706
DaZuiZui wants to merge 6 commits into
apache:masterfrom
DaZuiZui:fix-window-function-over-identity

Conversation

@DaZuiZui
Copy link
Copy Markdown
Contributor

@DaZuiZui DaZuiZui commented May 18, 2026

Description

This PR fixes incorrect window function results when multiple RANK() expressions with different ORDER BY clauses are planned together and some window input symbols come through identity projections.

Fix window ordering symbol remapping

GatherAndMergeWindows now keeps each window node's ordering scheme aligned with the symbols produced by the merged child. When the rule merges windows across identity projections, it remaps the ordering keys instead of reusing stale symbols from the original node. This prevents a preceding window, such as RANK() OVER (ORDER BY time), from affecting the ordering used by following rank functions.

Before / After

The tables below are captured from real local CLI runs with the same data and SQL:

  • Before: latest origin/master commit 4ac0b7ee029587d7a6f3393dcac24ca53fc0b7c3
  • After: this PR branch fix-window-function-over-identity

Query:

SELECT "time", amplitude, close_hfq, ma_5, ma_64, ma_120,
  RANK() OVER (ORDER BY "time") AS rank_time,
  RANK() OVER (ORDER BY amplitude DESC) AS rank_amplitude,
  RANK() OVER (ORDER BY close_hfq DESC) AS rank_close_hfq,
  RANK() OVER (ORDER BY ma_5) AS rank_ma5,
  RANK() OVER (ORDER BY ma_64 DESC) AS rank_ma64,
  RANK() OVER (ORDER BY ma_120) AS rank_ma120
FROM quant.stock_kline_weekly
WHERE "time" >= 2026-04-11 AND symbol = '600000'
ORDER BY "time";

Before this PR, once rank_time appeared as the first window function, later rank functions could reuse the wrong ordering symbols after window merging. In the real run below, rank_amplitude incorrectly follows rank_time, while rank_ma64 and rank_ma120 incorrectly follow the opposite order.

time rank_time rank_amplitude rank_close_hfq rank_ma5 rank_ma64 rank_ma120
2026-04-17 1 1 1 5 5 5
2026-04-24 2 2 2 4 4 4
2026-05-01 3 3 3 3 3 3
2026-05-08 4 4 4 2 2 2
2026-05-15 5 5 4 1 1 1

After this PR, every rank function keeps its own ordering:

time rank_time rank_amplitude rank_close_hfq rank_ma5 rank_ma64 rank_ma120
2026-04-17 1 2 1 5 1 1
2026-04-24 2 1 2 4 2 2
2026-05-01 3 4 3 3 3 3
2026-05-08 4 5 4 2 4 4
2026-05-15 5 3 4 1 5 5

The reported ma_3, ma_13, and ma_120 case was also reproduced with a real CLI run. Before this PR, all following ranks matched rank_time:

time rank_time rank_ma3 rank_ma13_desc rank_ma120
2026-04-17 1 1 1 1
2026-04-24 2 2 2 2
2026-05-01 3 3 3 3
2026-05-08 4 4 4 4
2026-05-15 5 5 5 5

After this PR, each moving-average rank follows its own ORDER BY:

time rank_time rank_ma3 rank_ma13_desc rank_ma120
2026-04-17 1 5 1 1
2026-04-24 2 4 2 2
2026-05-01 3 3 3 3
2026-05-08 4 2 4 4
2026-05-15 5 1 5 5

Cover abnormal window ranking cases

Added integration tests for the reported table-model cases, including:

  • a leading RANK() OVER (ORDER BY time) followed by rank functions over other columns
  • mixed ascending and descending ordering keys
  • ma_3, ma_13, and ma_120 ranking cases where following ranks previously matched the first rank unexpectedly
  • tie handling for RANK() results

Fixes #17635


This PR has:

  • been self-reviewed.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Tested with:

  • ./mvnw clean package -P with-integration-tests -DskipTests -ntp
  • ./mvnw clean package -pl distribution -am -DskipTests -ntp
  • ./mvnw clean verify -DskipUTs -Dit.test=IoTDBWindowFunction3IT -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -PTableSimpleIT -P with-integration-tests -ntp
  • ./mvnw clean verify -DskipUTs -Dit.test=IoTDBWindowFunction3IT#* -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -PTableSimpleIT -P with-integration-tests -ntp
  • real before/after local CLI verification using rebuilt standalone IoTDB distributions
    • Before output captured from latest origin/master (4ac0b7ee029587d7a6f3393dcac24ca53fc0b7c3) at /tmp/iotdb-main-window-rank.out
    • After output captured from this PR branch at /tmp/iotdb-after-window-rank.out

Key changed/added classes (or packages if there are too many classes) in this PR
  • org.apache.iotdb.db.queryengine.plan.relational.planner.iterative.rule.GatherAndMergeWindows
  • org.apache.iotdb.relational.it.db.it.IoTDBWindowFunction3IT

@DaZuiZui DaZuiZui changed the title Fix window function ordering with identity projections [IOTDB-17635] Fix incorrect ordering of multiple window RANK functions May 18, 2026
@HTHou HTHou requested a review from Wei-hao-Li May 18, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] SELECT中存在多个 Window RANK 时,除第一个以外的RANK排序错误

1 participant