test: promote try_to_date/try_to_timestamp SQL tests to native coverage#4973
Open
u70b3 wants to merge 2 commits into
Open
test: promote try_to_date/try_to_timestamp SQL tests to native coverage#4973u70b3 wants to merge 2 commits into
try_to_date/try_to_timestamp SQL tests to native coverage#4973u70b3 wants to merge 2 commits into
Conversation
…verage Investigation of apache#4556 showed the reported fallback does not exist on current main: in non-ANSI mode Spark's ReplaceExpressions expands try_to_date/try_to_timestamp to the exact same tree as to_date/ to_timestamp (Cast(..., LEGACY) or GetTimestamp(failOnError=false)), which Comet already accelerates. The 15 spark_answer_only markers were conservative annotations from apache#4555 (that mode only compares answers without asserting native execution), not measured fallbacks. Flip the 15 try_to_date/try_to_timestamp queries to default query mode (native coverage + answer check), keeping the 3 try_make_timestamp markers which belong to apache#4554. Verified on Spark 4.1: CometSqlFileTestSuite try_datetime passes with all queries running fully native. Also update the expression support matrix (try_to_date/try_to_timestamp 🔜 -> ✅) and add audit entries. Closes apache#4556
u70b3
force-pushed
the
test/promote-try-datetime-native
branch
from
July 18, 2026 04:32
9ae7f62 to
b8b08e5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #4556.
Rationale for this change
Investigation of #4556 showed that the reported fallback of
try_to_date/try_to_timestampdoes not exist on currentmain. All four functions (to_date,to_timestamp, and theirtry_variants) areRuntimeReplaceableand are expanded by Spark'sReplaceExpressionsbefore Comet sees the plan. In non-ANSI mode the expansions are identical between thetry_and non-try_forms:Cast(..., EvalMode.LEGACY)(EvalMode.fromBoolean(false)isLEGACY, notTRY)Cast(GetTimestamp(..., failOnError = false))Comet already accelerates both building blocks (native
Cast;GetTimestampvia the codegen dispatcher since #4454), so there is no serde gap to close. The 15spark_answer_onlymarkers intry_datetime.sqlwere conservative annotations introduced in #4555 — that mode only compares answers without asserting native execution — and predate theGetTimestampdispatch support.What changes are included in this PR?
try_datetime.sql: flip the 15try_to_date/try_to_timestampqueries fromspark_answer_onlyto the defaultquerymode (native coverage + answer check). The 3try_make_timestampmarkers are unchanged; they belong to try_make_timestamp returns wrong values for invalid inputs instead of NULL #4554.docs/.../expressions.md: support matrix entries fortry_to_date/try_to_timestamp🔜 -> ✅.docs/.../expression-audits/datetime_funcs.md: add audit entries for both functions.How are these changes tested?
passes with all 15 queries executing fully native (the default
querymode asserts an all-Comet plan and compares answers against Spark). This directly verifies the issue's acceptance criteria: native execution with NULL-on-invalid-input behavior matching Spark.