fix: read nested offsets of sliced arrays once when concatenating a native sort's payload

[msafonov]

Was stacked on #6. #6 is now merged into branch-1.1 and this PR is retargeted there, so it contains only its own commit.

Problem

On the night of 2026-10-09, 10 tasks of platform.mongo-uber-loader-daily failed on joom-1.1-4 in the native sort. All of them had wide nested Mongo rows (products, productVariants, parcels, trackings, orderJLShippingInfo, ...):

• CometNativeException: assertion failed: end <= self.len() (9 tasks)
• CometNativeException: range start index 17 out of range for slice of length 1 (platform_critical_jl-alerts_daily)

Stack: ArrayData::slice <- late_materialize::offsets_fit <- Gather::concatenate <- Gather::try_new (one task also went through ExternalSorter::sort_and_spill).

Cause

offsets_fit (added in 6975eba) walks ArrayData and applies the offset twice:

• ArrayData::buffer::<i32>(0) already starts at data.offset(), but the code indexed it again with [offset..=offset + len].
• ArrayData::slice of a struct already slices its children and keeps the offset, but the code sliced the children again by data.offset().

Top-level columns come from to_data() with offset 0, so they were fine. The panic needs a list, struct or map nested inside a list of a sliced input batch, which means value_offsets()[0] != 0 (for example, batches coming out of the coalescer). The existing test schema had no such nesting.

Fix

offsets_fit now walks typed arrays (as_list, as_map, as_struct, as_fixed_size_list). Their value offsets and columns already account for the slice. merge passes it the arrays directly.

Tests

• offsets_fit_reads_a_list_nested_in_a_sliced_list: panicked before the fix with range start index 17 out of range for slice of length 1.
• offsets_fit_reads_a_struct_nested_in_a_sliced_list: panicked before the fix with assertion failed: end <= self.len().
• nested_list_offsets_that_would_overflow_are_not_concatenated: the i32 overflow guard still works under nesting.
• sliced_nested_rows_match_the_reference: end to end through SortExec on sliced batches of list<struct<int, list<utf8>>> plus map<utf8, utf8>, both in memory and with a spill. It panicked before the fix and now matches the reference output.

Run locally (arm64 dev container):

• vendored datafusion-physical-plan: 1755 passed
• cargo test --workspace: 1918 passed, 0 failed
• cargo fmt and cargo clippy --workspace -D warnings: clean
• spotless:check and scalastyle: clean
• WideRowSortFallbackSuite: 17/17
• CometWindowExecSuite 71/71, CometFuzzTestSuite 45/45, CometExecSuite 155 passed / 2 canceled

🤖 Generated with Claude Code

[msafonov]

Local JVM suites (Spark 3.5, arm64 dev container, libcomet built from this commit): CometWindowExecSuite 71/71, CometFuzzTestSuite 45/45, CometExecSuite 155 succeeded / 0 failed / 2 canceled, WideRowSortFallbackSuite 17/17. spotless:check and scalastyle clean.

🤖 Generated with Claude Code