Skip to content

sort should not be removed when order by a + b where a and b are sorted as they can overflow and wrap #23902

Description

@rluvaton

Describe the bug

Sort should not be removed when doing addition (and other operators that marked as kept ordering in binary expression code) on 2 sorted columns since they can overflow and the default behavior is wrapping so the sort will be broken

To Reproduce

Have this data file data/uint8_overflow_order.csv:

a,b
1,1
2,3
200,10
255,10

and add this to order.slt for example

# Adding two ASC-ordered unsigned columns is only monotonically increasing while
# the sum does not overflow. `a + b` on UInt8 uses wrapping arithmetic, so the
# last row (255 + 10) wraps to 9 and breaks the ordering. A sort on the sum
# therefore must NOT be optimized away just because both inputs are ASC.
statement ok
set datafusion.execution.target_partitions = 1;

statement ok
CREATE EXTERNAL TABLE ordered_u8 (
  a TINYINT UNSIGNED NOT NULL,
  b TINYINT UNSIGNED NOT NULL
)
STORED AS CSV
LOCATION 'data/uint8_overflow_order.csv'
OPTIONS ('format.has_header' 'true')
WITH ORDER (a ASC)
WITH ORDER (b ASC);

query TT
EXPLAIN SELECT (a + b) AS result FROM ordered_u8 ORDER BY result ASC;
----
logical_plan
01)Sort: result ASC NULLS LAST
02)--Projection: ordered_u8.a + ordered_u8.b AS result
03)----TableScan: ordered_u8 projection=[a, b]
physical_plan DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/data/uint8_overflow_order.csv]]}, projection=[a@0 + b@1 as result], file_type=csv, has_header=true

query I
SELECT (a + b) AS result FROM ordered_u8 ORDER BY result ASC;
----
2
5
210
9

statement ok
DROP TABLE ordered_u8;

# The SLT runner sets `target_partitions` to 4 instead of using the default, so
# reset it explicitly.
statement ok
set datafusion.execution.target_partitions = 4;

Expected behavior

The SortExec should not be removed as it break the ordering when overflow

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions