Distinguish OpenAI batch timeout and cancellation in deferrable tasks - #72149
Draft
Lee-W wants to merge 2 commits into
Draft
Distinguish OpenAI batch timeout and cancellation in deferrable tasks#72149Lee-W wants to merge 2 commits into
Lee-W wants to merge 2 commits into
Conversation
A deferred OpenAITriggerBatchOperator reported every non-success outcome as the same OpenAIBatchJobException, so a task that merely ran out of patience looked identical to a batch that failed, and neither could be handled separately. The synchronous path has always raised OpenAIBatchTimeout for the same condition. The trigger now reports why it stopped, and the resuming task picks the matching exception from that field rather than from the message text.
The synchronous path cancels the batch before raising a timeout, but the deferrable path only failed the task and left the batch running and billing on OpenAI's side. A deferred timeout now requests cancellation with the batch id carried by the trigger event. Cancellation is asynchronous, so the batch reports "cancelling" for a while before it settles; a cancel that fails is logged and never masks the timeout. Deliberately not cancelling on a polling_error: batches.cancel is irreversible, and the failure mode there is unknown and often a transient, Airflow-side error rather than a real batch problem. OpenAI's batch has a 24-hour completion window that bounds it on its own, so leaving it alone costs a bounded amount; wrongly cancelling it is unrecoverable data loss. Anthropic cancels on its equivalent error branch, but its session has no such natural end point, so that precedent does not apply here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A deferred OpenAITriggerBatchOperator reported every non-success outcome as the
same OpenAIBatchJobException, so a task that merely ran out of patience looked
identical to a batch that failed, and neither could be handled separately. The
synchronous path has always raised OpenAIBatchTimeout for the same condition.
The trigger now reports why it stopped, and the resuming task picks the matching
exception from that field rather than from the message text.
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.