Skip to content

Make OpenAI Responses token ceilings templated and surface truncation - #72150

Draft
Lee-W wants to merge 1 commit into
apache:mainfrom
astronomer:openai-response-token-ceilings
Draft

Make OpenAI Responses token ceilings templated and surface truncation#72150
Lee-W wants to merge 1 commit into
apache:mainfrom
astronomer:openai-response-token-ceilings

Conversation

@Lee-W

@Lee-W Lee-W commented Aug 27, 2026

Copy link
Copy Markdown
Member

Dags need a per-run token ceiling that can vary by environment, but max_output_tokens and max_tool_calls were only reachable through the non-templated response_kwargs dict. Promote both to first-class, templated operator arguments, coerce the rendered string to a positive int before the request goes out, and reject an invalid or non-positive value instead of silently running with no ceiling. Also reject the same key being set both as an argument and inside response_kwargs, so one silently wins.

OpenAI's Responses API has no monetary cost limit to expose, so this is a token ceiling only -- for a cost cap, use
apache-airflow-providers-common-ai instead. Hitting max_output_tokens does not fail the request; the response comes back incomplete with truncated output text, which the previous 'may be empty' warning did not describe. Surface
incomplete_details.reason in the log message so operators can tell a truncation apart from other non-completed statuses.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Dags need a per-run token ceiling that can vary by environment, but
max_output_tokens and max_tool_calls were only reachable through the
non-templated response_kwargs dict. Promote both to first-class,
templated operator arguments, coerce the rendered string to a
positive int before the request goes out, and reject an invalid or
non-positive value instead of silently running with no ceiling.
Also reject the same key being set both as an argument and inside
response_kwargs, so one silently wins.

OpenAI's Responses API has no monetary cost limit to expose, so this
is a token ceiling only -- for a cost cap, use
apache-airflow-providers-common-ai instead. Hitting
max_output_tokens does not fail the request; the response comes
back incomplete with truncated output text, which the previous
'may be empty' warning did not describe. Surface
incomplete_details.reason in the log message so operators can tell
a truncation apart from other non-completed statuses.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant