Add JitteredCronTimetable for deterministic schedule jitter#69705
Draft
Pebble32 wants to merge 2 commits into
Draft
Add JitteredCronTimetable for deterministic schedule jitter#69705Pebble32 wants to merge 2 commits into
Pebble32 wants to merge 2 commits into
Conversation
Signed-off-by: Adam <111773160+Pebble32@users.noreply.github.com>
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
Signed-off-by: Adam <111773160+Pebble32@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
JitteredCronTimetable, an opt-inCronTriggerTimetablesubclass that shifts each DAG's fire time by a deterministic, per-DAG offset drawn from[0, max_jitter). This spreads out DAGs that share a cron expression so they no longer all fire at the same instant, without changinglogical_date/data_intervalsemantics.Why
@dailyexpands to0 0 * * *, so every daily DAG in a deployment is scheduled at exactly midnight. In large deployments this "thundering herd" at the cron boundary overloads the scheduler and workers — enough to cause task failures when dozens of DAGs are born at the same instant.The existing ways to deal with this don't actually de-collide the schedule:
@dailyintentparallelism,max_active_tasks_per_dag, pool slots)JitteredCronTimetableis the only approach that moves the fire times themselves, deterministically: the sameseed(e.g. the DAG id) always maps to the same offset, so runs stay stable and predictable across scheduler restarts and timetable serialization. Motivated by the discussion in #69027.What
JitteredCronTimetable(CronTriggerTimetable)in both the Task SDK (author-facing, attrs-based) and airflow-core (scheduler-side), plus the serialization wiring (BUILTIN_TIMETABLESmapping,serialize/deserialize, encode/decode across the SDK↔core boundary).CronTriggerTimetable:seed: strandmax_jitter: timedelta.md5(seed) % max_jitter.total_seconds(), applied as a "strip → cron → apply" coordinate shift so cron/DST alignment is fully delegated to the parent and only the wall-clock fire time is shifted.seed="",max_jitter=timedelta(0)) the offset is zero and it behaves identically toCronTriggerTimetable. Nothing changes for anyone who doesn't use it.Tests
airflow-core/tests/unit/timetables/test_jittered_cron_timetable.py, modeled on the existingtest_trigger_timetable.py:America/New_York);max_jitterreproducesCronTriggerTimetableexactly (catchup on/off);[0, max_jitter), and spread across distinct seeds;serialize/deserializeround-trips seed + window + derived offset;Was generative AI tooling used to co-author this PR?
Generated-by: Claude (Claude Code), following the guidelines
related: #69027