Repository navigation
fix(bigquery-jdbc): request int64 timestamps so the REST path is lossless - #14890
Open
keshavdandeva wants to merge 3 commits into
Open
keshavdandeva wants to merge 3 commits into
keshavdandeva wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request implements support for lossless int64-encoded timestamps when picosecond precision is disabled. It configures DataFormatOptions to request int64 timestamps by default, preserves FieldValue wrappers for TIMESTAMP columns to identify these encoded values, and decodes them to epoch microseconds during type conversion and range formatting. The review feedback suggests improving the robustness of the range bound parsing in BigQueryTemporalUtility by wrapping the Long.valueOf conversions in a try-catch block to handle potential NumberFormatExceptions and throw a BigQueryJdbcException.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
b/577457377
Summary
On the REST path, BigQuery returns
TIMESTAMPvalues as float64 seconds by default. That loses microseconds for dates far from 1970, and9999-12-31 23:59:59.999999 UTCcomes back as year10000. The value is rounded by the server, so the driver can't fix it on the client.Fix
useInt64Timestamp(true), which returns exact epoch microseconds. WithEnableTimestampPicos=1we still requestISO8601_STRING, because BigQuery rejects both together.TIMESTAMPvalues as epoch micros, as the Arrow path already does. This covers top-level, STRUCT/ARRAY and RANGE values.Tests
BigQueryConnectionTest: default connections request int64 timestamps.BigQueryJsonResultSetTest: int64TIMESTAMPandRANGE<TIMESTAMP>values decode exactly.ITTimestampTimeZoneTest: adds9999-12-31 23:59:59.999999 UTCto the existinggetTimestamp/getStringchecks across 4 JVM zones. These checks fail without the fix (+10000-01-01) and pass with it.