Commit 23404e8
### Problem
When using streaming responses, some models or API endpoints may return
`usage` fields (`prompt_tokens`, `completion_tokens`, `total_tokens`) as
`None` or omit them entirely. The current implementation passes these
values directly to the `ResponseUsage` Pydantic model, which expects
integers. This causes a validation error:
3 validation errors for ResponseUsage
input_tokens
Input should be a valid integer [type=int_type, input_value=None,
input_type=NoneType]
output_tokens
Input should be a valid integer [type=int_type, input_value=None,
input_type=NoneType]
total_tokens
Input should be a valid integer [type=int_type, input_value=None,
input_type=NoneType]
### Solution
This PR ensures that all token fields passed to `ResponseUsage` are
always integers. If any of the fields are `None` or missing, they
default to `0`. This is achieved by using `or 0` and explicit `is not
None` checks for nested fields.
**Key changes:**
- All `input_tokens`, `output_tokens`, `total_tokens` fields use `or 0`
fallback.
### Impact
- Fixes Pydantic validation errors for streaming responses with
missing/None usage fields.
- Improves compatibility with OpenAI and third-party models.
- No breaking changes; only adds robustness.
fixes #1179
Co-authored-by: thomas <[email protected]>
1 parent bc32b99 commit 23404e8
1 file changed
+3
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
493 | 493 | | |
494 | 494 | | |
495 | 495 | | |
496 | | - | |
497 | | - | |
498 | | - | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
499 | 499 | | |
500 | 500 | | |
501 | 501 | | |
| |||
0 commit comments