Skip to content

[Fix] Costs miss Fast mode inference usage - #1630

Merged
mrubens merged 4 commits into
developfrom
fix/fast-inference-accounting-2tukv8ghenf27
Aug 25, 2026
Merged

[Fix] Costs miss Fast mode inference usage#1630
mrubens merged 4 commits into
developfrom
fix/fast-inference-accounting-2tukv8ghenf27

Conversation

@roomote-roomote

@roomote-roomote roomote-roomote Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

​Opened on behalf of @mrubens. Follow up by mentioning @roomote-roomote, in the web UI, or in Slack.

What changed

  • Persist completed Fast parent-turn messages and advisor/judge child-session messages from the OpenCode session tree.
  • Use the final parent assistant event as an ordering barrier so every earlier current-turn parent and child event has entered persistence before Fast returns, with a one-second deadline when the long-lived event stream omits that event.
  • Keep an exceptional bounded reconciliation fallback scoped to the current parent message and child sessions created in the current turn, preserving the existing non-task:<source>:<session>:<message> idempotency key and original participant ownership.
  • Add focused coverage for intermediate parent usage, advisor/judge usage, duplicate events, shared-conversation ownership, bounded fallback work, Fast service opt-in, and Costs analytics inclusion.

Why this change was made

Fast mode previously recorded only the final parent response. Intermediate parent turns and advisor/judge sessions incurred inference cost but were omitted from llm_usage_events, causing Costs analytics to undercount Fast usage.

Impact

Costs now includes the complete successful Fast turn without rewriting historical usage ownership or doing work proportional to the warm conversation's lifetime. OpenCode 1.18.10's AssistantMessage / message.updated contract does not expose a stable provider request or generation identifier, so this PR does not guess one from untyped part metadata. The precise reconciliation follow-up is to expose that identifier in OpenCode's assistant-message event contract, then persist it in usage-event details (or dedicated columns if it becomes a query key) for matching against OpenRouter generation exports.

Validation

  • Cloud Agents targeted Vitest: 98 passed.
  • Web Costs targeted Vitest: 4 passed.
  • Cloud Agents and Web package TypeScript checks passed.
  • Web ESLint and touched-file oxfmt/oxlint checks passed.
  • Full pre-push oxlint, residual lint, fast typecheck, and knip gates passed.

@roomote-community

roomote-community Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

No new commits since the last review. See task

  • Reconciliation rewrites historical usage ownership in shared Fast conversations (packages/cloud-agents/src/server/non-task-provider-usage.ts:1032).
  • Each Fast turn re-fetches and re-upserts the entire warm session history, causing quadratic work (packages/cloud-agents/src/server/non-task-provider-usage.ts:1011).
  • A missing final usage event waits forever on the long-lived event stream, preventing bounded reconciliation (packages/cloud-agents/src/server/non-task-provider-usage.ts:1017).

Reviewed 61ad44d

Comment thread packages/cloud-agents/src/server/non-task-provider-usage.ts Outdated
Comment thread packages/cloud-agents/src/server/non-task-provider-usage.ts Outdated
@roomote-roomote

roomote-roomote Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

Fixed the remaining long-lived event-stream wait in 9ac3ba9: the final-event barrier now expires after one second, clears its waiter, and runs the bounded current-turn fallback. Regression coverage uses a non-terminating stream that omits the final event. Targeted tests pass (98), package typecheck and touched-file checks pass, and the full pre-push gates pass.

Comment thread packages/cloud-agents/src/server/non-task-provider-usage.ts
@mrubens
mrubens marked this pull request as ready for review August 25, 2026 13:18
- Bound and cancel fallback reconciliation calls: session.messages and
  session.children now carry an abort signal linked to the prompt's
  controller plus a 5s reconciliation deadline, so a wedged OpenCode
  server can no longer stall the reply or hold the leased server.
- Discover child sessions for the fallback via session.children instead
  of event-stream bookkeeping, so a mid-turn stream failure cannot hide
  subagent usage, and bound them to sessions created in the current turn
  so a warm shared conversation's historical usage is never re-recorded
  under the current requester.
- Warn when the final message has no parent id and intermediate parent
  usage cannot be reconciled.
- Drain in-flight usage recordings in the prompt finally block so event-
  driven writes settle on error paths too and late recordings cannot
  outlive the call.
- Hoist the event-monitor gate into needsEventMonitor so the subscribe
  and race sites cannot drift.
@mrubens
mrubens merged commit 0d38d34 into develop Aug 25, 2026
16 checks passed
@mrubens
mrubens deleted the fix/fast-inference-accounting-2tukv8ghenf27 branch August 25, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant