Repository navigation
Round the next fetch dates without a Calendar - #2290
Merged
Merged
Conversation
DateUtils.round builds a GregorianCalendar for every URL, which was the largest share of the CPU used by AbstractStatusUpdaterBolt.execute(). Rounding to the second or minute is now done on the epoch time, half up, which gives the same dates as DateUtils. Rounding to the hour depends on the time zone and still goes through DateUtils. DateUtils can also move a date in the hour repeated when the clocks go back to its other occurrence, an hour away. The new rounding keeps it where it is. Signed-off-by: Julien Nioche <julien@digitalpebble.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rzo1
approved these changes
Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AbstractStatusUpdaterBolt.execute()rounds the next fetch date withDateUtils.round, which builds aGregorianCalendarfor every URL. When profiling the status updater withStatusUpdaterBenchmarkgoing throughexecute(), this was the largest share of the CPU used by the bolt threads, about a third of it, more than the cache or the filtering of the metadata.Rounding to the second or the minute is now done on the epoch time, half up, which gives the same dates as
DateUtils. A quick timing gives about 230 ns per call withDateUtilsand a few ns without it. Rounding to the hour depends on the time zone, e.g. with an offset of 5:30, so it still goes throughDateUtils.The tests compare the new rounding with
DateUtilson edge values and on 100,000 random dates. They pass with UTC, Asia/Kolkata, America/St_Johns, Asia/Kathmandu, Europe/Paris, America/New_York and Australia/Lord_Howe as default time zone.While testing, I found that
DateUtilscan move a date in the hour repeated when the clocks go back to the other occurrence of that hour. In Paris, 02:22:47 CEST is rounded to the minute as 02:23 CET, an hour later. The new rounding keeps it at 02:23 CEST, a test covers this case.🤖 Generated with Claude Code