Skip to content

fix(langchain): handle empty llm generations - #1839

Open
arpansahu wants to merge 1 commit into
langfuse:mainfrom
arpansahu:fix/langchain-tool-input-parsing
Open

fix(langchain): handle empty llm generations#1839
arpansahu wants to merge 1 commit into
langfuse:mainfrom
arpansahu:fix/langchain-tool-input-parsing

Conversation

@arpansahu

@arpansahu arpansahu commented Aug 27, 2026

Copy link
Copy Markdown

Problem

CallbackHandler.on_llm_end() assumes every LangChain LLMResult contains at least one generation in the last generation batch. If a provider returns an empty generation batch but still includes model and usage metadata, the callback raises IndexError, logs the error, and never ends the generation span.

Reproducer

handler.on_llm_start(
    {"name": "EmptyLLM"},
    ["hello"],
    run_id=run_id,
    invocation_params={"model_name": "empty-model"},
)
handler.on_llm_end(
    LLMResult(
        generations=[[]],
        llm_output={"token_usage": {"prompt_tokens": 2, "completion_tokens": 0, "total_tokens": 2}},
    ),
    run_id=run_id,
)
# before: IndexError is logged and no span is exported
# after: span ends with output=None and preserved usage/model metadata

Fix

Extract the last available generation by skipping empty generation batches, and use None as the output when no generation is present. This keeps the span lifecycle correct while preserving existing output extraction for normal chat and text generations.

Testing

  • python -m pytest tests\\unit\\test_langchain.py::test_llm_callback_ends_generation_with_empty_generations -q failed before the fix and passed after.
  • python -m pytest tests\\unit\\test_langchain.py -q -> 24 passed.
  • ruff format langfuse\\langchain\\CallbackHandler.py tests\\unit\\test_langchain.py --check -> passed after formatting.
  • ruff check langfuse\\langchain\\CallbackHandler.py tests\\unit\\test_langchain.py -> All checks passed.

Note: full offline tests\\unit baseline still has pre-existing Windows/environment failures unrelated to this change (test_path, prompt subprocess missing opentelemetry, and prompt mock errors).

Greptile Summary

The PR makes LangChain LLM completion handling tolerate empty generation batches so generation spans can still finish with model and usage metadata.

  • Moves LLM result extraction into a helper that selects the last available generation and returns None when none exists.
  • Adds regression coverage for an empty generation result carrying model and token-usage metadata.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness, security, or quality issues identified.

The empty-result path now avoids indexing an empty batch, safely passes None through the observation update, preserves model and usage metadata, and ends the generation span.

Reviews (1): Last reviewed commit: "fix(langchain): handle empty llm generat..." | Re-trigger Greptile

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: arpan sahu <28574248+arpansahu@users.noreply.github.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@arpansahu

Copy link
Copy Markdown
Author

Disclosure: this contribution was prepared with AI assistance (GitHub Copilot CLI), and
I am flagging that explicitly rather than leaving you to guess.

The defect, the fix and the regression test were verified locally: the new test fails on
unmodified upstream and passes with the change applied, and the surrounding suite is green.

If your project would prefer not to take AI-assisted contributions, or you would rather this
were reworked and resubmitted by hand, please just close it -- no objection at all, and
apologies for the noise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants