Anthropic - Adopt ToolCallingManager API #2228
Conversation
- Update AnthropicChatModel to use the new ToolCallingManager API, while ensuring full API backward compatibility. - Introduce Builder to instantiate a new AnthropicChatModel since the number of overloaded constructors is growing too big. - Update documentation about tool calling and Anthropic support for that. Signed-off-by: Thomas Vitale <[email protected]>
| if (!isProxyToolCalls(prompt, this.defaultOptions) && this.isToolCall(chatResponse, Set.of("tool_use"))) { | ||
| var toolCallConversation = handleToolCalls(prompt, chatResponse); | ||
| return this.internalStream(new Prompt(toolCallConversation, prompt.getOptions()), chatResponse); | ||
| if (ToolCallingChatOptions.isInternalToolExecutionEnabled(prompt.getOptions()) && chatResponse.hasToolCalls() && chatResponse.hasFinishReasons(Set.of("tool_use"))) { |
There was a problem hiding this comment.
For Anthropic in streaming mode, checking for tool calls is not enough. We also need to check for the finish reason.
That was not necessary for OpenAI, Ollama and Mistral, for which it's enough to check for tool calls.
There was a problem hiding this comment.
Yeh, I remember that there were a cases requiring the finish reason. we have to review the bedrock-converse and the vertex gemini refactoring again.
There was a problem hiding this comment.
Yeah, I think we can add some auto-tests in those special cases, also as a way to document the behavior (ensuring tools are only called once).
I thought about adding some additional tests here for Anthropic, but we first need this defect fixed: #1370
|
Thanks @ThomasVitale |
2 participants
Relates to gh-2207