Fix reciprocal rank fusion, streaming deltas, and error handling

[sroussey]

Summary

This PR fixes critical bugs in reciprocal rank fusion (RRF), streaming event accumulation, and error handling across multiple AI tasks and providers. It also adds Ollama structured generation support and improves robustness in knowledge base search and tool calling.

Key Changes

Reciprocal Rank Fusion (RerankerTask)

• Fixed RRF implementation: Now correctly fuses two independent rankings (retrieval scores + lexical keyword overlap) instead of just reranking by position. The query is now passed to reciprocalRankFusion() to enable keyword-based ranking.
• Extracted keywordMatchCount() helper: Shared by both simpleRerank() and reciprocalRankFusion() to avoid duplication and ensure consistent keyword matching logic.
• Improved RRF algorithm: Properly computes scoreRank and keywordRank separately, then combines them via 1/(k+rank_a) + 1/(k+rank_b) so chunks ranked highly by both signals float to the top.

Streaming Event Accumulation (StreamEventAccumulator)

• Removed mixed-mode rejection: Text and object deltas on distinct ports now coexist and compose into a single output object (e.g., tool-calling streams text on text and tool calls on toolCalls).
• Fixed delta precedence: Accumulated deltas now take precedence over the finish payload, preventing structural default scaffolds (e.g., { text: "", toolCalls: [] }) from clobbering streamed content.
• Mirrors task-graph behavior: Ensures .run() and streaming produce identical output.

Tool Calling (Ollama & ToolCallingTask)

• Ollama tool calling: Moved tool call validation to the delta emission point (not finish), so hallucinated tool names are filtered before streaming to consumers.
• Structural finish scaffold: Ollama now emits only { text: "", toolCalls: [] } on finish, letting the accumulator's delta precedence rule preserve streamed calls.
• Session disposal timing: ToolCallingTask.execute() and executeStream() now register the session disposer before calling super.execute() / super.executeStream(), ensuring cleanup fires even if the provider throws or the stream aborts mid-iteration.

Knowledge Base Search Error Handling (AiChatWithKbTask)

• Wrapped KB search in try-catch: A single KB's embedding/search failure no longer aborts the entire turn. Errors are logged and that KB returns empty results, allowing other KBs to still contribute.

Ollama Structured Generation

• New Ollama_StructuredGeneration.ts: Implements JSON-mode structured generation for Ollama, passing the schema as the format parameter and emitting object-delta events with progressively parsed partial JSON.

Gemini Text Generation

• Extracted buildGenerationConfig() helper: Maps canonical sampling params (maxTokens, temperature, topP, frequencyPenalty, presencePenalty) onto Gemini's generationConfig, only setting defined fields so callers that omit a param keep the provider's default.

AiChatTask

• Empty assistant message handling: A turn that produces no assistant content (provider emitted only non-text events, an immediate finish, or an empty/aborted response) no longer appends an empty message to history nor counts as a completed turn, preventing context poisoning.

Error Classification (AiJob)

• Improved HTTP status detection: The regex for extracting HTTP status codes from error messages now requires HTTP-shaped context (e.g., "HTTP 503", "status: 429") to avoid misclassifying bare numbers like "512" in "Sequence length 512 exceeds limit" or model IDs.

Tool Call Parsing (ToolCallParsers)

• Fixed Llama parser offset: Now tracks the actual block start position instead of reconstructing the offset from a literal {"name": " prefix, which breaks when the model's output format varies.

Provider Registration (AiProvider)

• **Worker cleanup on re-

https://claude.ai/code/session_015j321YUsXJRVEVkBmEavtg

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	62.5%	25322 / 40512
🔵	Statements	62.34%	26206 / 42035
🔵	Functions	63.75%	4814 / 7551
🔵	Branches	51.08%	12366 / 24206

File Coverage

No changed files found.

Generated in workflow #2622 for commit cc880f5 by the Vitest Coverage Report Action

[Copilot]

Pull request overview

This PR fixes several correctness issues and edge cases across the AI stack—most notably reciprocal-rank-fusion reranking, streaming output accumulation semantics, and provider/session cleanup—while adding Ollama JSON-mode (structured generation) support and improving robustness around KB search and error classification.

Changes:

• Correct RRF reranking by fusing independent retrieval-score and keyword-overlap rankings, with shared keyword matching logic.
• Align streaming accumulation behavior with task-graph semantics (compose deltas across ports; deltas take precedence over structural finish scaffolds) and update tests accordingly.
• Extend provider capabilities (Ollama json-mode), improve provider/task cleanup and error-handling (worker termination on re-register, KB search isolation, safer HTTP status extraction, parser offset fix).

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
providers/ollama/src/ai/common/Ollama_ToolCalling.ts	Emits validated tool-call deltas during streaming and uses a structural finish scaffold.
providers/ollama/src/ai/common/Ollama_StructuredGeneration.ts	Adds Ollama JSON-mode structured generation streaming run-fn with partial JSON parsing + final object finish payload.
providers/ollama/src/ai/common/Ollama_JobRunFns.ts	Registers the new Ollama json-mode run-fn in the worker runtime.
providers/ollama/src/ai/common/Ollama_JobRunFns.browser.ts	Registers the new Ollama json-mode run-fn in the browser runtime.
providers/ollama/src/ai/common/Ollama_CapabilitySets.ts	Adds ["text.generation","json-mode"] capability set.
providers/ollama/src/ai/common/Ollama_Capabilities.ts	Includes json-mode in inferred capability lists.
providers/google-gemini/src/ai/common/Gemini_TextGeneration.ts	Introduces buildGenerationConfig() to map canonical sampling params without forcing undefined values.
packages/test/src/test/ai/StreamEventAccumulator.test.ts	Updates/expands tests for composing deltas across distinct ports and finish-scaffold precedence.
packages/test/src/test/ai/collectStream.test.ts	Updates collectStream expectations for mixed text/object deltas across ports.
packages/test/src/test/ai-provider-api/OllamaProvider.test.ts	Updates provider capability/run-fn assertions to include json-mode registration.
packages/ai/src/task/ToolCallingTask.ts	Registers session disposal earlier to ensure cleanup even on throw/abort mid-execution/stream.
packages/ai/src/task/RerankerTask.ts	Fixes RRF algorithm and factors out shared keyword match counting.
packages/ai/src/task/DocumentEnricherTask.ts	Tightens schema/types for required inputs (doc_id, documentTree) and removes unnecessary casts.
packages/ai/src/task/base/AiTask.ts	Clarifies and centralizes capability gating semantics via modelMeetsRequires.
packages/ai/src/task/AiChatWithKbTask.ts	Wraps per-KB search in try/catch so one KB failure degrades to empty results instead of aborting the turn.
packages/ai/src/task/AiChatTask.ts	Prevents appending an empty assistant message when a turn yields no assistant text.
packages/ai/src/provider/AiProvider.ts	Ensures worker-backed providers terminate existing workers on re-register and on partial-registration failure cleanup.
packages/ai/src/provider-utils/ToolCallParsers.ts	Fixes Llama parser content slicing by tracking the true balanced-block start offset.
packages/ai/src/job/AiJob.ts	Tightens HTTP-status extraction regex to avoid misclassifying bare numbers in error messages.
packages/ai/src/capability/StreamEventAccumulator.ts	Allows composing text+object deltas across ports and ensures accumulated deltas take precedence over finish scaffolds.
packages/ai/README.md	Documents execution/retry behavior (no automatic retries for default in-process strategies).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sroussey]

Added in cc880f5 — packages/test/src/test/ai-provider-api/Ollama_StructuredGenerationStream.test.ts, a fake-stream unit test mirroring Ollama_TextGenerationStream.test.ts. It covers: the output schema passed through as Ollama's format, progressive object-delta emission on the object port from partial JSON, finish.data.object carrying the fully parsed object, partial-JSON fallback on truncated output, pre-aborted-signal rejection (before any client call/emit), and a single stream.abort() on mid-iteration abort. The factory is now exported from @workglow/ollama/ai-runtime so the test can import it.

---

Generated by Claude Code