Skip to content

fix: stop Agent retries after output limit - #12303

Open
Nikhi00718 wants to merge 2 commits into
deepset-ai:mainfrom
Nikhi00718:codex/fix-agent-output-limit-exit
Open

fix: stop Agent retries after output limit#12303
Nikhi00718 wants to merge 2 commits into
deepset-ai:mainfrom
Nikhi00718:codex/fix-agent-output-limit-exit

Conversation

@Nikhi00718

@Nikhi00718 Nikhi00718 commented Aug 11, 2026

Copy link
Copy Markdown

Related Issues

Proposed Changes:

  • Map OpenAI Responses API completions stopped by max_output_tokens to Haystack's existing length finish reason in both streaming and non-streaming paths.
  • Treat a tool-call-free assistant reply with that finish reason as a text Agent exit, while retaining the retry behavior for other empty replies such as discarded malformed tool calls.
  • Add sync, async, streaming, and non-streaming regression coverage.

How did you test it?

  • C:\Program Files\Hatch\hatch.exe run test:unit test/components/agents/test_agent.py test/components/generators/chat/test_openai_responses_conversion.py (121 passed, 4 integration tests deselected)
  • C:\Program Files\Hatch\hatch.exe run test:unit test/components/generators/chat (251 passed, 43 integration tests deselected)
  • C:\Program Files\Hatch\hatch.exe run test:types
  • C:\Program Files\Hatch\hatch.exe run ruff check --fix
  • C:\Program Files\Hatch\hatch.exe run ruff format
  • C:\Program Files\Hatch\hatch.exe run pre-commit run --files ... (all scoped hooks passed)

Notes for the reviewer

The new Agent regression tests fail on main because the Agent makes a second model call after a length-truncated empty response. The Responses conversion regressions also fail on main because the same condition is exposed as stop in streaming mode and has no normalized finish reason in non-streaming mode.

hatch run fmt is not directly executable on Windows because the repository script's Unix-style semicolon is passed literally to Ruff; the two configured Ruff commands above were run separately through Hatch.

AI assistance disclosure

I identified and reproduced the issue, then used OpenAI Codex to assist with the implementation and regression-test work. I personally reviewed the resulting diff and ran the validation commands listed above to verify the fix. Any full-suite or local-environment limitations are documented in the validation section.

Checklist

  • I have read the contributors guidelines.
  • I have updated the related issue with helpful information about this PR.
  • I have added unit tests and documentation, if applicable.
  • I have followed the commit conventions.
  • I have documented any new or changed behavior.
  • I have added a release note.
  • I have run the pre-commit hooks.

@vercel

vercel Bot commented Aug 11, 2026

Copy link
Copy Markdown

@Nikhi00718 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@Nikhi00718
Nikhi00718 marked this pull request as ready for review August 11, 2026 10:12
@Nikhi00718
Nikhi00718 requested a review from a team as a code owner August 11, 2026 10:12
@Nikhi00718
Nikhi00718 requested review from julian-risch and removed request for a team August 11, 2026 10:12
@CLAassistant

CLAassistant commented Aug 11, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@julian-risch
julian-risch requested review from sjrl and removed request for julian-risch August 13, 2026 07:27
@julian-risch

Copy link
Copy Markdown
Member

@sjrl Could you please take over this review? Thank you!

Comment thread haystack/components/generators/chat/openai_responses.py Outdated
Comment thread haystack/components/generators/chat/openai_responses.py
Comment thread releasenotes/notes/agent-exit-on-output-limit-3ca9a92c399520df.yaml Outdated
Comment thread releasenotes/notes/agent-exit-on-output-limit-3ca9a92c399520df.yaml Outdated
@sjrl

sjrl commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

@Nikhi00718 please also handle the merge conflicts with main

@Nikhi00718
Nikhi00718 force-pushed the codex/fix-agent-output-limit-exit branch from d6a340d to 5285cb8 Compare August 14, 2026 17:09
@Nikhi00718

Copy link
Copy Markdown
Author

Addressed all review feedback:

  • Handle both response.completed and response.incomplete terminal streaming events.
  • Normalize max_output_tokens to length and content_filter to content_filter for streaming chunks and ChatMessage metadata.
  • Updated the release note to describe the generic Agent behavior, the reasoning-token exhaustion scenario, and the FinishReason metadata.
  • Updated coverage to use the real ResponseIncompleteEvent and test both incomplete reasons.
  • Confirmed the branch is up to date with main and mergeable.

Verification:

  • 126 passed, 2 skipped across the two affected test modules.
  • Ruff lint and format checks pass for all affected files.
  • Targeted mypy check for the updated conversion test passes.

All four review threads have been replied to and resolved.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @Nikhi00718, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Aug 14, 2026
@HaystackBot
HaystackBot marked this pull request as draft August 14, 2026 18:35
@sjrl
sjrl marked this pull request as ready for review August 14, 2026 18:42
@sjrl sjrl removed the cla-pending PR is in draft until the contributor signs the CLA label Aug 14, 2026
@github-actions github-actions Bot added the type:documentation Improvements on the docs label Aug 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/agents
  agent.py 159
  haystack/components/generators/chat
  openai_responses.py 716
  haystack/core/pipeline
  pipeline.py
Project Total  

This report was generated by python-coverage-comment-action

@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Aug 14, 2026
@HaystackBot
HaystackBot marked this pull request as draft August 14, 2026 19:09
@Nikhi00718
Nikhi00718 marked this pull request as ready for review August 14, 2026 19:19
@Nikhi00718

Copy link
Copy Markdown
Author

@sjrl The CLA Assistant says all committers have signed, but the license/cla status is missing on the latest commit and the cla-pending label remains. Could you please retrigger the CLA check and review assignment?

@HaystackBot
HaystackBot marked this pull request as draft August 14, 2026 19:37
@Nikhi00718
Nikhi00718 marked this pull request as ready for review August 14, 2026 20:15
@HaystackBot
HaystackBot marked this pull request as draft August 14, 2026 20:29
@sjrl

sjrl commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

@sjrl The CLA Assistant says all committers have signed, but the license/cla status is missing on the latest commit and the cla-pending label remains. Could you please retrigger the CLA check and review assignment?

Yeah sorry about that. I'll be able to finish reviewing this early next week and look into the bot issue.

@HaystackBot
HaystackBot marked this pull request as ready for review August 15, 2026 01:52
@HaystackBot

Copy link
Copy Markdown
Contributor

Thanks for signing the CLA, @Nikhi00718! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

@HaystackBot HaystackBot removed the cla-pending PR is in draft until the contributor signs the CLA label Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent loops to max_agent_steps when a reasoning model returns no text because it hits max_output_tokens

5 participants