Skip to content

Use upstream_inference_cost for OpenRouter BYOK cost calculationΒ #5144

@chrarnoldus

Description

@chrarnoldus

What specific problem does this solve?

I recently made a PR that corrects the displayed cost when OpenRouter bring-your-own-key (BYOK) is used: #4543

OpenRouter has since added a new field upstream_inference_cost (https://openrouter.ai/docs/use-cases/usage-accounting#cost-breakdown) that contains an estimate of the upstream cost. Using this field will likely be a more reliable and robust solution than hardcoding the 20x multiplier. Additionally, the cached token count display should be restored to give users full insight in their spend.

We already made this change in Kilo Code: Kilo-Org/kilocode#836, I will be opening a PR to upstream these changes shortly.

Additional context (optional)

No response

Request checklist

  • I've searched existing Issues and Discussions for duplicates
  • This describes a specific problem with clear impact and context

Interested in implementing this?

  • Yes, I'd like to help implement this feature

Implementation requirements

  • I understand this needs approval before implementation begins

How should this be solved? (REQUIRED if contributing, optional otherwise)

Add the upstream_inference_cost to the cost, instead of multiplying the latter by 20x.

How will we know it works? (Acceptance Criteria - REQUIRED if contributing, optional otherwise)

I will share some screenshots and calculations showing that the numbers are reasonable.

Technical considerations (REQUIRED if contributing, optional otherwise)

Very small localized change.

Trade-offs and risks (REQUIRED if contributing, optional otherwise)

I thinks the risks are very low.

Metadata

Metadata

Assignees

Labels

Issue - In ProgressSomeone is actively working on this. Should link to a PR soon.enhancementNew feature or requestproposal

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions