[AURON #2510] Add non-UTC timestamp coverage for native datediff

[Sigma-Ma]

Which issue does this PR close?

Closes #2510

Rationale for this change

The datediff coverage added in #2502 only uses DATE inputs. TIMESTAMP inputs also need coverage for conversion to dates in the session time zone.

What changes are included in this PR?

Add a regression test using TIMESTAMP columns in America/Los_Angeles. It covers local and UTC date boundaries, positive and negative differences, and null inputs.

The test checks explicit expected results, compares with Spark, and verifies native operators.

Are there any user-facing changes?

No. This change only adds regression coverage.

How was this patch tested?

On Spark 3.5.8 with Scala 2.12 and JDK 8:

./build/mvn -B -ntp -Ppre -Pspark-3.5 -Pscala-2.12 -pl spark-extension-shims-spark -DskipBuildNative -DskipTests=false '-Dsuites=org.apache.auron.AuronFunctionSuite datediff' test
./build/mvn -B -ntp -Ppre -Pspark-3.5 -Pscala-2.12 -pl spark-extension-shims-spark -DskipBuildNative -DskipTests=false -Dsuites=org.apache.auron.AuronFunctionSuite test
./dev/reformat --check

Was this patch authored or co-authored using generative AI tooling?

• Yes
• No

Generated-by: OpenAI Codex (GPT-6)

[slfan1989]

Thanks for adding this coverage.

The test cases clearly distinguish session-local date boundaries from UTC boundaries and also cover signed and null results.

However, the latest CI is consistently failing for Spark 4.0, 4.1, and 4.2, while the Spark 3.x jobs pass.

Could you please investigate the Spark 4.x failures and either fix the compatibility issue or confirm that they are unrelated before merging?

[Sigma-Ma]

Thanks for adding this coverage.

The test cases clearly distinguish session-local date boundaries from UTC boundaries and also cover signed and null results.

However, the latest CI is consistently failing for Spark 4.0, 4.1, and 4.2, while the Spark 3.x jobs pass.

Could you please investigate the Spark 4.x failures and either fix the compatibility issue or confirm that they are unrelated before merging?

Thanks for pointing this out. checkSparkAnswerAndOperator already collects the Auron DataFrame, and the subsequent checkAnswer(df, ...) collected the same AQE plan again. Spark 4.x hits an internal assertion during that second execution.

I updated the explicit expected-result check to use a fresh DataFrame, while keeping the Spark comparison and native operator assertion. The focused test passes locally with Spark 4.0 and Scala 2.13. Spark 4.1 and 4.2 will be verified by CI.