Jbbrown/bedrock cost calculations #1573
Conversation
…into jbbrown/bedrock_cost_calculations * jbbrown/aws_custom_arn_for_intelligent_prompt_routing: Bedrock specific cost calculation, including support for intelligent prompt routing
|
dosubot
bot
added
size:XL
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
| if (invokedModelId) { | ||
| // Extract the model name from the ARN | ||
| // Example ARN: arn:aws:bedrock:us-west-2:699475926481:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0 | ||
| const modelMatch = invokedModelId.match(/\/([^\/]+)(?::|$)/) |
There was a problem hiding this comment.
The ARN model extraction logic is duplicated here and in cost.ts. Consider extracting this into a shared utility function to avoid code duplication and ensure consistency.
| | "openai-native" | ||
| | "vertex" | ||
| | "unbound" | ||
| | "glama" |
There was a problem hiding this comment.
There is a potential typographical error on line 8 in the provider union where 'glama' is used. Considering that elsewhere in the code (in model costs) the term 'llama' is used, please verify if 'glama' was intended or if it should be corrected to 'llama'.
| | "glama" | |
| | "llama" |
1 participant
Bedrock Cost Calculations with Intelligent Prompt Routing Support
Overview
This PR adds support for accurate cost calculations for AWS Bedrock models, with special handling for intelligent prompt routing. The implementation enables precise tracking of costs when using AWS Bedrock's intelligent prompt router feature, which dynamically selects the most appropriate model for each request.
Changes
New Features
cost.tsfile that centralizes all cost calculation logic for different providersTechnical Improvements
ApiStreamUsageChunkinterface to include theinvokedModelIdfield for tracking which model was actually usedWhy These Changes Are Needed
Testing
All tests pass successfully, including:
Future Considerations
Important
Adds cost calculation module for AWS Bedrock models with intelligent prompt routing and updates interfaces and tests for accurate cost tracking.
cost.tsmodule for centralized cost calculation logic.ApiStreamUsageChunkinstream.tsnow includesinvokedModelId.cost.test.tsfor various models and scenarios.bedrock.tsfor custom ARN issues.This description was created by
for 886e5db. It will automatically update as commits are pushed.