Skip to content

[Fix] Costs miss Fast mode inference usage - #1630

Draft
roomote-roomote[bot] wants to merge 3 commits into
developfrom
fix/fast-inference-accounting-2tukv8ghenf27
Draft

[Fix] Costs miss Fast mode inference usage#1630
roomote-roomote[bot] wants to merge 3 commits into
developfrom
fix/fast-inference-accounting-2tukv8ghenf27

Conversation

@roomote-roomote

@roomote-roomote roomote-roomote Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

​Opened on behalf of @mrubens. Follow up by mentioning @roomote-roomote, in the web UI, or in Slack.

What changed

  • Persist completed Fast parent-turn messages and advisor/judge child-session messages from the OpenCode session tree.
  • Use the final parent assistant event as an ordering barrier so every earlier current-turn parent and child event has entered persistence before Fast returns, with a one-second deadline when the long-lived event stream omits that event.
  • Keep an exceptional bounded reconciliation fallback scoped to the current parent message and child sessions created in the current turn, preserving the existing non-task:<source>:<session>:<message> idempotency key and original participant ownership.
  • Add focused coverage for intermediate parent usage, advisor/judge usage, duplicate events, shared-conversation ownership, bounded fallback work, Fast service opt-in, and Costs analytics inclusion.

Why this change was made

Fast mode previously recorded only the final parent response. Intermediate parent turns and advisor/judge sessions incurred inference cost but were omitted from llm_usage_events, causing Costs analytics to undercount Fast usage.

Impact

Costs now includes the complete successful Fast turn without rewriting historical usage ownership or doing work proportional to the warm conversation's lifetime. OpenCode 1.18.10's AssistantMessage / message.updated contract does not expose a stable provider request or generation identifier, so this PR does not guess one from untyped part metadata. The precise reconciliation follow-up is to expose that identifier in OpenCode's assistant-message event contract, then persist it in usage-event details (or dedicated columns if it becomes a query key) for matching against OpenRouter generation exports.

Validation

  • Cloud Agents targeted Vitest: 98 passed.
  • Web Costs targeted Vitest: 4 passed.
  • Cloud Agents and Web package TypeScript checks passed.
  • Web ESLint and touched-file oxfmt/oxlint checks passed.
  • Full pre-push oxlint, residual lint, fast typecheck, and knip gates passed.

@roomote-community

roomote-community Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

No code issues found. See task

  • Reconciliation rewrites historical usage ownership in shared Fast conversations (packages/cloud-agents/src/server/non-task-provider-usage.ts:1032).
  • Each Fast turn re-fetches and re-upserts the entire warm session history, causing quadratic work (packages/cloud-agents/src/server/non-task-provider-usage.ts:1011).
  • A missing final usage event waits forever on the long-lived event stream, preventing bounded reconciliation (packages/cloud-agents/src/server/non-task-provider-usage.ts:1017).

Reviewed 9ac3ba9

Comment thread packages/cloud-agents/src/server/non-task-provider-usage.ts Outdated
Comment thread packages/cloud-agents/src/server/non-task-provider-usage.ts Outdated
@roomote-roomote

roomote-roomote Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

Fixed the remaining long-lived event-stream wait in 9ac3ba9: the final-event barrier now expires after one second, clears its waiter, and runs the bounded current-turn fallback. Regression coverage uses a non-terminating stream that omits the final event. Targeted tests pass (98), package typecheck and touched-file checks pass, and the full pre-push gates pass.

Comment thread packages/cloud-agents/src/server/non-task-provider-usage.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant