Summary
In @tanstack/ai@0.28.0, fallbackStructuredOutputStream (used when an adapter doesn't implement structuredOutputStream — e.g. AnthropicTextAdapter) calls adapter.structuredOutput(), gets back { data, rawText, usage }, but discards result.usage entirely. Downstream consumers reading token usage from either RUN_FINISHED.usage or structured-output.complete.value.usage see undefined, even though the adapter correctly populated the field from the provider response.
Reproduction
Any call shaped like:
```ts
import { chat, EventType } from "@tanstack/ai";
import { AnthropicTextAdapter } from "@tanstack/ai-anthropic";
const stream = chat({
adapter: new AnthropicTextAdapter({ apiKey, baseURL }, "claude-3-5-sonnet"),
systemPrompts: ["..."],
messages: [{ role: "user", content: "..." }],
outputSchema: mySchema,
stream: true,
});
for await (const chunk of stream) {
if (chunk.type === EventType.RUN_FINISHED) {
console.log(chunk.usage); // undefined
}
if (chunk.type === "CUSTOM" && chunk.name === "structured-output.complete") {
console.log(chunk.value.usage); // undefined
}
}
```
The actual provider response includes usage (verified via `console.log(response.usage)` inside `AnthropicTextAdapter.structuredOutput()` — `input_tokens`, `output_tokens`, `cache_read_input_tokens` all present), and `buildAnthropicUsage()` produces a complete `TokenUsage` object, but it never reaches consumers.
Root cause
In `packages/ai/src/activities/chat/index.ts` (built artifact: `dist/esm/activities/chat/index.js` around line 1607 of `@tanstack/ai@0.28.0`):
```js
const result = await adapter.structuredOutput(options); // result.usage is populated here
yield {
type: EventType.CUSTOM,
name: "structured-output.complete",
value: { object: result.data, raw: result.rawText }, // result.usage dropped
model,
timestamp,
};
yield {
type: EventType.RUN_FINISHED,
runId,
threadId,
model,
timestamp,
finishReason: "stop", // no usage field
};
```
Both yields drop `result.usage` despite the adapter's `structuredOutput` return contract including it.
Suggested fix
```diff
yield {
type: EventType.CUSTOM,
name: "structured-output.complete",
- value: { object: result.data, raw: result.rawText },
- value: { object: result.data, raw: result.rawText, usage: result.usage },
model,
timestamp,
};
yield {
type: EventType.RUN_FINISHED,
runId,
threadId,
model,
timestamp,
- finishReason: "stop",
- usage: result.usage,
};
```
Both touch points needed for backward compat with consumers that read either chunk shape.
Note: `StructuredOutputCompleteEvent.value` type in `types.d.ts:1009` would need `usage?: TokenUsage` added; `RunFinishedEvent.usage` already exists at `types.d.ts:781` (just unused on this path).
Impact
Any cost-tracking / observability layer that relies on `usage` in fallback path streams reports zero/null token counts. Pricing snapshots write but token counts are NULL → cost computation = 0. We hit this on DeepSeek (Anthropic endpoint) where every structured-output call goes through fallback because `AnthropicTextAdapter` doesn't implement `structuredOutputStream`.
Local patch via `pnpm patch` confirmed fix: token counts now flow end-to-end (verified with `promptTokens=125`, `completionTokens=1346`, `promptTokensDetails.cachedTokens=5760` reaching consumer).
Environment
- `@tanstack/ai@0.28.0`
- `@tanstack/ai-anthropic@0.15.1`
- Adapter: `AnthropicTextAdapter` against DeepSeek's `/anthropic` compatible endpoint
- Node 20.x
Related: #390 (middleware bypass in same fallback path), #526 (request for native streaming structured output in openrouter adapter — would also bypass this fallback when implemented).
Summary
In
@tanstack/ai@0.28.0,fallbackStructuredOutputStream(used when an adapter doesn't implementstructuredOutputStream— e.g.AnthropicTextAdapter) callsadapter.structuredOutput(), gets back{ data, rawText, usage }, but discardsresult.usageentirely. Downstream consumers reading token usage from eitherRUN_FINISHED.usageorstructured-output.complete.value.usageseeundefined, even though the adapter correctly populated the field from the provider response.Reproduction
Any call shaped like:
```ts
import { chat, EventType } from "@tanstack/ai";
import { AnthropicTextAdapter } from "@tanstack/ai-anthropic";
const stream = chat({
adapter: new AnthropicTextAdapter({ apiKey, baseURL }, "claude-3-5-sonnet"),
systemPrompts: ["..."],
messages: [{ role: "user", content: "..." }],
outputSchema: mySchema,
stream: true,
});
for await (const chunk of stream) {
if (chunk.type === EventType.RUN_FINISHED) {
console.log(chunk.usage); // undefined
}
if (chunk.type === "CUSTOM" && chunk.name === "structured-output.complete") {
console.log(chunk.value.usage); // undefined
}
}
```
The actual provider response includes usage (verified via `console.log(response.usage)` inside `AnthropicTextAdapter.structuredOutput()` — `input_tokens`, `output_tokens`, `cache_read_input_tokens` all present), and `buildAnthropicUsage()` produces a complete `TokenUsage` object, but it never reaches consumers.
Root cause
In `packages/ai/src/activities/chat/index.ts` (built artifact: `dist/esm/activities/chat/index.js` around line 1607 of `@tanstack/ai@0.28.0`):
```js
const result = await adapter.structuredOutput(options); // result.usage is populated here
yield {
type: EventType.CUSTOM,
name: "structured-output.complete",
value: { object: result.data, raw: result.rawText }, // result.usage dropped
model,
timestamp,
};
yield {
type: EventType.RUN_FINISHED,
runId,
threadId,
model,
timestamp,
finishReason: "stop", // no usage field
};
```
Both yields drop `result.usage` despite the adapter's `structuredOutput` return contract including it.
Suggested fix
```diff
yield {
type: EventType.CUSTOM,
name: "structured-output.complete",
model,
timestamp,
};
yield {
type: EventType.RUN_FINISHED,
runId,
threadId,
model,
timestamp,
};
```
Both touch points needed for backward compat with consumers that read either chunk shape.
Note: `StructuredOutputCompleteEvent.value` type in `types.d.ts:1009` would need `usage?: TokenUsage` added; `RunFinishedEvent.usage` already exists at `types.d.ts:781` (just unused on this path).
Impact
Any cost-tracking / observability layer that relies on `usage` in fallback path streams reports zero/null token counts. Pricing snapshots write but token counts are NULL → cost computation = 0. We hit this on DeepSeek (Anthropic endpoint) where every structured-output call goes through fallback because `AnthropicTextAdapter` doesn't implement `structuredOutputStream`.
Local patch via `pnpm patch` confirmed fix: token counts now flow end-to-end (verified with `promptTokens=125`, `completionTokens=1346`, `promptTokensDetails.cachedTokens=5760` reaching consumer).
Environment
Related: #390 (middleware bypass in same fallback path), #526 (request for native streaming structured output in openrouter adapter — would also bypass this fallback when implemented).