Fix token usage / cost often being underreported

[chrarnoldus]

Kilo Code PR: Kilo-Org/kilocode#1447

Description

Roo Code often underreports the total cost of a task, because the cost of some requests are missing. For example:

The first $0.00677 request is missing its cost in the UI, while requests 2-4 are not. Also the total cost (should be $0.0816) is off.

This is because a request is abandoned before the usage block has been processed when an LLM tries to use multiple tools (only one is allowed in the rules):

Roo-Code/src/core/task/Task.ts

Line 1430 in 714fafd

if (this.didAlreadyUseTool) {

(this code is inherited from Cline)

Test Procedure

Run tasks with a variety of providers and models and verify all requests have a cost reported. Kimi K2 seems to be especially affected. Sonnet 4 sometimes, Gemini 2.5 Pro not so much.

I did test this with OpenRouter and costs are reported more reliably than before. Also tested with Anthropic and Ollama for regressions.

Pre-Submission Checklist

• Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
• Scope: My changes are focused on the linked issue (one major feature/fix per PR).
• Self-Review: I have performed a thorough self-review of my code.
• Testing: New and/or updated tests have been added to cover my changes (if applicable).
• Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
• Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

In this screenshot all costs are reported and the tally is correct ($0.0591):

Additional Notes

I'll add some comments to the code.

Get in Touch

Christiaan in shared Slack

---

Important

Fixes underreporting of token usage and cost in Task.ts by processing all request chunks and updating usage data even if requests are interrupted.

• Behavior:
  • Fixes underreporting of token usage and cost in Task class in Task.ts by ensuring all request chunks are processed.
  • Adds drainStreamInBackgroundToFindAllUsage() to process remaining chunks and update usage data even if a request is interrupted.
• Functions:
  • Modifies recursivelyMakeClineRequests() to handle incomplete requests by processing all chunks in the background.
  • Updates updateApiReqMsg() to include all token and cost data.
• Misc:
  • Removes redundant TelemetryService call for capturing LLM completion in the main loop, now handled in the background task.

^{This description was created by for 60f0a03. You can customize this summary. It will automatically update as commits are pushed.}

[chrarnoldus]

The comment calls out OpenRouter specifically, but I think if seen usage being misreported with other providers as well.

[daniel-lxs]

I've pushed some improvements to address the review comments:

• Added proper error handling for the background promise
• Fixed the race condition by using local variables in the background task
• Made the timeout configurable via a constant in global-settings.ts
• Improved code organization

One concern I noticed: with this implementation, when a user cancels a request, the background stream continues running to collect usage data. This differs from the original behavior where streams would be immediately terminated.

Should we keep this new behavior for better usage tracking, or would it be better to add a check to stop the background task immediately when this.abort is true? The OpenRouter docs mention stream cancellation best practices here: https://openrouter.ai/docs/api-reference/streaming#stream-cancellation

[chrarnoldus]

Should we keep this new behavior for better usage tracking, or would it be better to add a check to stop the background task immediately when this.abort is true?

If it is decided to abort the stream I do feel there should be a visual indication that the cost figure is no longer reliable. Same goes for the timeout. (I can make a PR for this.)

[daniel-lxs]

@chrarnoldus
Thank you for the review! I believe we should keep the old functionality of aborting the stream for now and in a future PR we can add a visual indicator, the main concern is that this would cause the API costs to some users just to report the correct cost data.

Let me know what you think!

[chrarnoldus]

Ok, it's just that not every provider supports cancellation so costs could rack up anyway.

[chrarnoldus]

@daniel-lxs can you consider taking this if we make the timeout very small? Then it shouldn't impact cost much. I can make a PR for the visual indicator afterwards.

[daniel-lxs]

@chrarnoldus I can give this the green light and let the team decide if this is something they want, can you take a look at the failing CI workflow?

[chrarnoldus]

@daniel-lxs I fixed the initial failure, I don't think the new one is related to the changes in this PR.