fix: ensure prompt_tokens is always present in Anthropic streaming responses #1509
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes an issue where
prompt_tokenswas missing from streaming responses for Anthropic models (both direct Anthropic API and via Vertex AI), causing billing/usage tracking to be incorrect.Root cause: When streaming, the
message_startevent may not always includeinput_tokens. The code relied solely on this event to setprompt_tokens, resulting inundefinedvalues that were omitted from the JSON response.The fix:
input_tokensin themessage_deltaevent as a fallback (per Anthropic docs, these are cumulative counts)0if neither source provides the valueprompt_tokensin the usage object to ensure it's always presentChanges
src/providers/google-vertex-ai/chatComplete.ts- Vertex AI Claude streamingsrc/providers/anthropic/chatComplete.ts- Direct Anthropic streamingTest plan
/v1/chat/completions- verifyprompt_tokensis present/v1/chat/completions- verifyprompt_tokensis presenttotal_tokenscalculation is still correctBefore (problematic response)
After (fixed response)