Skip to content

Conversation

@mrubens
Copy link
Collaborator

@mrubens mrubens commented Mar 11, 2025

Context

As described in #1227, some APIs return the usage counts in every chunk, causing our current method of tracking to show completely inaccurate counts (and likely confusing the sliding window calculations).

Implementation

@tbphp had a great suggestion here to just use the last usage if there are multiple: #1227 (comment)

Screenshots

before after
Screenshot 2025-03-10 at 11 02 04 PM Screenshot 2025-03-10 at 11 01 56 PM

How to Test

I signed up for the SiliconFlow provider at https://siliconflow.cn/ and used the free deepseek-ai/DeepSeek-R1-Distill-Qwen-7B model to test.


Important

Fixes usage tracking in OpenAiHandler by using only the last usage metrics from API streams, with comprehensive tests added.

  • Behavior:
    • Fixes usage tracking in OpenAiHandler by using only the last usage metrics from API streams.
    • Handles cases where usage data is only in the final chunk or not provided at all.
  • Testing:
    • Adds openai-usage-tracking.test.ts to test usage metrics handling with multiple chunks, final chunk only, and no usage data scenarios.
  • Implementation:
    • Modifies createMessage() in openai.ts to store and yield the last usage metrics.

This description was created by Ellipsis for f306461. It will automatically update as commits are pushed.

@changeset-bot
Copy link

changeset-bot bot commented Mar 11, 2025

🦋 Changeset detected

Latest commit: f306461

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
roo-cline Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 11, 2025
@dosubot dosubot bot added the bug Something isn't working label Mar 11, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 11, 2025
@mrubens mrubens merged commit ebb5a57 into main Mar 11, 2025
18 checks passed
@mrubens mrubens deleted the fix_open_ai_usage branch March 11, 2025 03:29
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Mar 11, 2025
@hannesrudolph
Copy link
Collaborator

I swear we had a PR or two handling this.

@mrubens
Copy link
Collaborator Author

mrubens commented Mar 11, 2025

Yeah #962 was another, I think we can close it now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants