Skip to content

Conversation

@cte
Copy link
Collaborator

@cte cte commented Jun 18, 2025

Description

Some providers aren't set up to yield costs in real-time. In some of these cases we sum the cost ourselves and yield a final usage with the total cost. For providers that don't do this there is a final fallback:

const updateApiReqMsg = (cancelReason?: ClineApiReqCancelReason, streamingFailedMessage?: string) => {
  this.clineMessages[lastApiReqIndex].text = JSON.stringify({
    ...JSON.parse(this.clineMessages[lastApiReqIndex].text || "{}"),
    tokensIn: inputTokens,
    tokensOut: outputTokens,
    cacheWrites: cacheWriteTokens,
    cacheReads: cacheReadTokens,
    cost:
      totalCost ??
      calculateApiCostAnthropic(
        this.api.getModel().info,
        inputTokens,
        outputTokens,
        cacheWriteTokens,
        cacheReadTokens,
      ),
    cancelReason,
    streamingFailedMessage,
  } satisfies ClineApiReqInfo)
}

This incorrectly assumes that the total cost calculation is the Anthropic variant, and it also happens after we emit the LLM Completion event that Roo Code Cloud uses.

The option that makes sense to me is to have our provider classes emit a usage with the total cost. Here's the fix for the Anthropic provider and I'm having Roo Code audit all of the others.


Important

Ensure Anthropic provider yields usage data with total cost by updating AnthropicHandler and Task classes.

  • Behavior:
    • AnthropicHandler in anthropic.ts now accumulates token usage and yields a final usage event with totalCost using calculateApiCostAnthropic().
    • Task in Task.ts updates telemetry capture to calculate cost using calculateApiCostAnthropic() if totalCost is undefined.
  • Tests:
    • Increase test timeout in ShadowCheckpointService.spec.ts from 10,000 to 20,000 milliseconds.

This description was created by Ellipsis for 53f048c. You can customize this summary. It will automatically update as commits are pushed.

@cte cte requested review from jr and mrubens as code owners June 18, 2025 18:20
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working labels Jun 18, 2025
@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Jun 18, 2025
import { RepoPerTaskCheckpointService } from "../RepoPerTaskCheckpointService"

vitest.setConfig({ testTimeout: 10_000 })
vitest.setConfig({ testTimeout: 20_000 })
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is flaking for the win32 GHA runner.

cost: totalCost,
cost:
totalCost ??
calculateApiCostAnthropic(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we'd require all providers to emit a totalCost, but there's no enforcement mechanism for that right now, so we'll re-use this fallback.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jun 18, 2025
@cte cte merged commit 775457c into main Jun 18, 2025
13 checks passed
@cte cte deleted the cte/yield-cost-in-anthropic-provider branch June 18, 2025 18:59
@github-project-automation github-project-automation bot moved this from Triage to Done in Roo Code Roadmap Jun 18, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Jun 18, 2025
cte added a commit that referenced this pull request Jun 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

5 participants