Commit 2f85765
fix: support tool_choice with specific tool names in LiteLLM streaming (fixes #1846)
This change fixes a Pydantic validation error that occurred when using
LiteLLM with streaming enabled and specifying a specific tool name for
tool_choice parameter.
Problem:
When users specified tool_choice="my_tool_name" with streaming enabled,
the SDK would incorrectly cast it to Literal["auto", "required", "none"],
causing a Pydantic validation error.
The issue was in litellm_model.py line 376, where the Response object was
created with an incorrect type cast:
tool_choice=cast(Literal["auto", "required", "none"], tool_choice)
However, tool_choice can be:
- A Literal: "auto", "required", "none"
- A ChatCompletionNamedToolChoiceParam dict with specific tool name
- The Converter.convert_tool_choice() already handles string tool names
Solution:
- Import ToolChoiceFunction from openai.types.responses
- Properly convert ChatCompletionNamedToolChoiceParam to ToolChoiceFunction
- Handle all valid tool_choice types when creating Response object
The fix ensures that when tool_choice is a dict like:
{"type": "function", "function": {"name": "my_tool"}}
It gets correctly converted to:
ToolChoiceFunction(type="function", name="my_tool")
Testing:
- Linting (ruff check) - passed
- Type checking (mypy) - passed
- Formatting (ruff format) - passed
Generated with Lucas Wang<[email protected]>
Co-Authored-By: Claude <[email protected]>1 parent 748ac80 commit 2f85765
1 file changed
+29
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| 27 | + | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| |||
32 | 33 | | |
33 | 34 | | |
34 | 35 | | |
| 36 | + | |
35 | 37 | | |
36 | 38 | | |
37 | 39 | | |
| |||
367 | 369 | | |
368 | 370 | | |
369 | 371 | | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
370 | 398 | | |
371 | 399 | | |
372 | 400 | | |
373 | 401 | | |
374 | 402 | | |
375 | 403 | | |
376 | | - | |
377 | | - | |
378 | | - | |
| 404 | + | |
379 | 405 | | |
380 | 406 | | |
381 | 407 | | |
| |||
0 commit comments