Skip to content

fix(compute): detect temporal rounding overflow - #1127

Open
fallintoplace wants to merge 9 commits into
apache:mainfrom
fallintoplace:fix/temporal-rounding-overflow
Open

fix(compute): detect temporal rounding overflow#1127
fallintoplace wants to merge 9 commits into
apache:mainfrom
fallintoplace:fix/temporal-rounding-overflow

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

Temporal rounding can overflow while converting timestamps to nanoseconds, calculating the rounding interval, or multiplying a rounded quotient. The current unchecked arithmetic can return an incorrect timestamp.

What changes are included in this PR?

Use checked arithmetic for temporal conversions, interval setup, rounding results, calendar conversion, and origin adjustments. Propagate overflow as an Arrow invalid-value error.

Are these changes tested?

  • go test ./arrow/compute

Are there any user-facing changes?

Inputs and options that cannot be represented during temporal rounding now return an error instead of a wrapped result.

@fallintoplace
fallintoplace force-pushed the fix/temporal-rounding-overflow branch 2 times, most recently from 65f4e12 to 71e18ac Compare August 7, 2026 21:01
@zeroshade

Copy link
Copy Markdown
Member

Since this is a draft, i'll hold off on further review until it's marked ready and the conflicts are resolved

@fallintoplace
fallintoplace force-pushed the fix/temporal-rounding-overflow branch from a61e0eb to 1c9b885 Compare August 23, 2026 21:12
@fallintoplace
fallintoplace marked this pull request as ready for review August 23, 2026 21:12

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found three blocking calendar-rounding issues: valid nanosecond results are rejected because helper boundaries are range-checked, wider temporal types are unnecessarily narrowed to nanoseconds, and long calendar periods still silently overflow through time.Duration midpoint arithmetic.


This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. After you've addressed the points above and pushed an update, an Apache Arrow Go maintainer — a real person — will take the next look at the PR. If you think one of the findings is misapplied, please reply on the PR and a maintainer will weigh in.

More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.

if dateErr != nil {
return 0, dateErr
}
yearEnd, dateErr := checkedCalendarDate(nextYear, 1, tz)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: Restricting this intermediate period boundary to the nanosecond timestamp window rejects inputs whose selected result is representable. For timestamp[ns] 2262-01-05 rounded half-up to one year, the correct result is 2262-01-01, but this checks the unused 2263-01-01 boundary and returns overflow. Likewise, 2201-01-01 rounded to 100 years should return 2200-01-01, but checking the 2300 boundary fails. Main succeeds in both cases. Please permit calendar boundaries outside the output window and validate only the selected result with timeToNanos; cover year, quarter, and month boundaries.

}

// convertToNanos converts a timestamp value to nanoseconds.
func convertToNanos(ts int64, unit arrow.TimeUnit) (int64, error) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: Every calendar path is still narrowed to an int64 nanosecond intermediate, even when the input and correct result are representable in their original type. For example, flooring Date32(1500-06-15) to one year should return 1500-01-01, but this PR returns overflow. The same affects Date64 and second/millisecond/microsecond timestamps, whose ranges are much wider than timestamp nanoseconds. Please perform calendar arithmetic without narrowing to nanoseconds and range-check the final result in the input unit.

if dateErr != nil {
return 0, dateErr
}
rounded = halfRoundPeriod(t, yearStart, yearEnd)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: halfRoundPeriod computes periodEnd.Sub(periodStart)/2; time.Time.Sub saturates at math.MaxInt64 for periods longer than about 292 years. Consequently, rounding 1948-01-01 to a 300-year multiple uses a midpoint in 1946 instead of the true 1950 midpoint and returns 2100 rather than 1800. This is pre-existing, but it is an unchecked calendar-rounding overflow directly within this PR’s scope. Please compute calendar midpoints without converting the entire period to time.Duration, with a regression test for a period over 292 years.

@fallintoplace
fallintoplace force-pushed the fix/temporal-rounding-overflow branch from e7f637f to e267708 Compare August 25, 2026 19:43

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update. The earlier nanosecond-boundary and long-period midpoint examples are fixed, and the test suite passes locally. I found four remaining overflow/range cases that still need correction before this is safe across the supported temporal ranges.


// Slow path: convert to nanoseconds for calendar origin or incompatible units
tsNanos := convertToNanos(ts, inputUnit)
tsNanos, err := convertToNanos(ts, inputUnit)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: The fixed-duration slow path still rejects representable wider-unit values when the requested rounding is finer than the input resolution. For example, flooring a second-resolution 1500-06-15 12:34:56 timestamp to one nanosecond is an exact no-op, but this conversion returns invalid: temporal rounding overflow. Please avoid narrowing when the input is already aligned to the requested interval, and add coverage outside the timestamp-nanosecond window.

return time.Time{}, overflowError()
}

startSeconds := periodStart.Unix()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: time.Time.Unix() is undefined once a boundary lies outside the int64 seconds range, so this still rejects a representable selected result at the edge of timestamp[s]. With an input of January 2 in the year containing math.MaxInt64 seconds, half-up rounding to one year should select that year's representable January 1; the next-year boundary exceeds int64, these calls wrap, and checkedSubInt64 returns overflow. Please compute the midpoint without requiring both boundaries to fit Unix seconds and cover this edge.

epochWeekStart = time.Date(epochWeekStart.Year(), epochWeekStart.Month(), epochWeekStart.Day(), 0, 0, 0, 0, tz)

daysSinceEpochWeek := int(startOfWeek.Sub(epochWeekStart).Hours() / 24)
daysSinceEpochWeek := int64(startOfWeek.Sub(epochWeekStart).Hours() / 24)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: This reintroduces the same approximately 292-year saturation through time.Time.Sub. Flooring a second-resolution 1500-06-15 12:34:56 timestamp to one week returns 1677-09-19 instead of 1500-06-10 because the day distance saturates before division. Please calculate calendar-day/week distance without time.Duration, with wide-range week regression coverage.

}

// Convert back to the input unit, validating only the selected result.
roundedTimestamp, err := arrow.TimestampFromTime(rounded, inputUnit)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: This does not actually validate second-resolution results: arrow.TimestampFromTime returns val.Unix() without a range check for arrow.Second. Ceiling math.MaxInt64 seconds to the next year therefore succeeds with wrapped value -9223372036852412416 instead of returning overflow. Please explicitly validate second-unit conversion (or make TimestampFromTime do so) and add a boundary test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants