Advancing Tool Support in Spring AI #2049
Description
This issue presents ideas and suggestions for consolidating and advancing tool support in Spring AI, before reaching the first GA version.
Tool Registration
Tools are modelled via the ToolCallback API. By default, Spring AI supports registering tools from methods and from functional objects.
Methods
In the context of Java, a tool can naturally be modelled as a method. Spring AI support registering methods as tools via the MethodToolCallback API.
Declarative tool registration is supported via the @Tool annotation on methods.
class MyTools {
@Tool(description = "Get the list of books written by the given author available in the library")
List<Book> booksByAuthor(String author) {
return bookService.getBooksByAuthor(new Author(author));
}
}@GetMapping("/chat")
String chat(String authorName) {
return chatClient.prompt()
.user("What books written by %s are available in the library?".formatted(authorName))
.tools(new MyTools())
.call()
.content();
}Functions
In special cases, there might be a wish for modelling tools as functional objects (Function/BiFunction/Supplier/Consumer). Spring AI support registering functional objects as tools via the FunctionToolCallback API.
@GetMapping("/chat")
String chat(String authorName) {
return chatClient.prompt()
.user("What books written by %s are available in the library?".formatted(authorName))
.tools(FunctionToolCallback.builder("booksByAuthor", author -> bookService().getBooksByAuthor(author))
.description("Get the list of books written by the given author available in the library")
.inputType(BookService.Author.class)
.build())
.call()
.content();
}Tool Execution
By default, tool execution is performed transparently by a ChatModel object using the ToolCallingManager API. In case of errors while executing tools, a ToolCallExceptionConverter is used to process the exception.
If you need full control over the tool execution, for example by asking the user permission before each tool execution, you can disable the internal tool execution feature and manage the tool calling workflow yourself using the ToolCallingManager API directly.
ChatOptions chatOptions = ToolCallingChatOptions.builder()
.toolCallbacks(ToolCallbacks.from(tools))
.internalToolExecutionEnabled(false)
.build();
Prompt prompt = new Prompt("What's the weather in Copenhagen?", chatOptions);
ChatResponse chatResponse = chatModel.call(prompt);
if (chatResponse.hasToolCalls()) {
List<Message> messages = toolCallingManager.executeToolCalls(prompt, chatResponse);
}
Prompt secondPrompt = new Prompt(messages, chatOptions);
ChatResponse secondChatResponse = chatModel.call(secondPrompt);Tool Resolution
Instead of providing tools as ToolCallbacks to ChatModel or ChatClient, you have the option to pass just tool names and let the framework resolve them into ToolCallbacks dynamically.
By default, Spring AI support dynamic tool resolution from the following:
- Functional objects registered as beans with the Spring context (
Function/BiFunction/Supplier/Consumer). ToolCallbacks registered as beans with the Spring context.
Additional resolution strategies can be provided via the ToolCallbackResolver API.
Example:
@Configuration(proxyBeanMethods = false)
class Functions {
@Bean
@Description("Get the list of books written by the given author available in the library")
public Function<Author, List<Book>> booksByAuthor(BookService bookService) {
return author -> bookService.getBooksByAuthor(author);
}
}