[pos-ei-don]

Description

With a reasoning_parser configured, streamed chat completions from the vLLM backend carry the model's reasoning in delta.content, and the final chunk repeats the answer and the reasoning a second time. Non-streaming responses are split correctly.

Reproduce with a thinking model (e.g. Qwen3 family, reasoning_parser: qwen3) and "stream": true with thinking enabled: the reasoning text arrives as choices[0].delta.content, delta.reasoning_content stays empty until the last chunk, which then contains the full reasoning and the full answer again. We saw the reasoning in content in 11 of 12 streamed requests with high reasoning effort.

Cause: the streaming path in _predict sends each raw text delta as ChatDelta.content (reasoning is only split after the loop, on the full text), and LocalAI forwards the backend's ChatDeltas as they are.

This PR uses vLLM's composed parser (ParserManager.get_parser, available since vLLM 0.30), the one vLLM's own OpenAI server streams with:

• one parser per streaming request, built with the request's chat_template_kwargs (so enable_thinking=false starts it in content mode) and tools,
• parse_delta per delta; reasoning, content and tool calls go out in their own fields as they are produced,
• the final chunk carries only token counts and logprobs.

If the parser raises mid-stream, the rest of the output is buffered and the final chunk sends what the full-text parse has beyond what was already streamed (suffixes per field and per tool-call index); where the full parse disagrees with what went out, nothing is guessed and the mismatch is logged.

Only active when a reasoning_parser is set. Without one, or on a vLLM without ParserManager, streaming is unchanged. get_parser is called without the tokenizer argument, which only exists in 0.32.

Notes for Reviewers

New server-less tests in backend/python/vllm/test.py (TestStreamingReasoningSplit, fake engine + fake parser like the existing TestStreamingToolParser): no reasoning in content and nothing repeated in the final chunk; parser failure mid-reasoning and after a partial tool call (rest of the arguments on the same index, no raw markup in content); unchanged raw streaming without ParserManager; chat_template_kwargs/tools forwarded. On unpatched master 3 of them fail (reasoning and a duplicated answer in content).

Run in the backend's venv against vLLM 0.31.0 and 0.32.0: new and existing server-less tests pass. Additionally checked outside the suite with vLLM's real qwen3 reasoning parser, qwen3_coder tool parser and a Qwen3 tokenizer, streaming a recorded completion in chunks of 1–4 tokens (as with speculative decoding): reasoning, content and tool calls separate correctly with thinking on and off, the final chunk is empty, and non-streaming output is unchanged.

Independent of #12872 (same file, no overlap; merges cleanly in either order).

Signed commits

• Yes, I signed my commits.
• Documentation updated (docs/content/) for user-facing changes, or not applicable

[localai-org-maint-bot]

@mudler source review complete against master 9c4a8b2a8: no blocking production-code findings. The composed parser separates reasoning/content/tool deltas, and the final reply avoids repeating text already emitted. Checked the parser calls against upstream vLLM 0.31.0 source.

I pushed c37aa56fa with two minor fixes: documentation of streaming behavior and its vLLM version limit, and an isolated request fixture for the fake-parser tests. On older vLLM profiles, _ParserRequest is absent; the original tests would silently fall back to raw streaming and fail their split assertions. The fixture removes that dependency. Older backend versions still retain their previous production streaming behavior.

Validation: clean integration with current master and whitespace checks pass. Runtime tests remain unverified: the available Python executables require a loader absent from this environment. Please validate the backend suite, including the new fake-parser tests on older profiles, before merging.

Source-review sign-off, conditional on backend test validation and human DCO certification of the added commit. It includes Assisted-by attribution and no AI Signed-off-by.