Handle functions as an agent

[habuma]

Expected Behavior

I want to be able to provide Spring AI with one or more functions that could be used to provide additional information or functionality. (See https://platform.openai.com/docs/guides/gpt/function-calling). Moreover, when the response to a prompt indicates that a function should be called, I'd like it if that decision, the call to the function, and the reissuing of a new prompt were all handled internally by Spring AI. This behavior is what I believe is commonly referred to as an "agent".

In short, I'd like to be able to configure Spring AI with one or more functions (perhaps as lambdas or method references) that could be used later. Then, if I ask a question that the LLM needs more information to answer, Spring AI would call those functions for me and resubmit the prompt. I don't want to have to do what is shown as "step 2" in the example at https://platform.openai.com/docs/guides/gpt/function-calling. That decision making should be handled internally by Spring AI.

My primary desire is to have function support, but almost as importantly, I want to not have to deal with the back and forth interactions in my own code. I want Spring AI to handle that internally.

Current Behavior

Spring AI does not yet support functions, so at this point there's no need for agent behavior. Currently Spring AI only handles one-turn Q&A type interactions.

Context

As an example (drawn from the aforementioned documentation), if I were to ask what the weather is in Boston, most/all LLMs wouldn't know that answer because they're not trained on real-time or even relatively current data. But if I provide a function that can lookup the weather in a city, then instead of the LLM responding that it doesn't know what the weather is, it could respond with an instruction to call the function. The application (or the agent on behalf of the application) would call that function as instructed, then submit a new prompt with weather data from which the LLM would be able to generate the desired answer.

[Kirbstomper]

I was bored at work so I played around a little bit with this. I am by no means an expert but I thought it would be cool if we could somehow auto configure and mark functions in the same way that Spring Cloud Functions does. I think this is the idea for "agents" right?

https://github.com/Kirbstomper/spring-ai/tree/function_playground

I created a new annotation and the code that should auto populate that. While on the client side its a bit quick and dirty but I have a integration test that works for using a function with the latest version of openai-java

[markpollack]

Yes, functions are on our roadmap @habuma Sort of gathering thoughts together to make a road map and will be creating issues and such in the coming days.

@Kirbstomper Thanks so much for the contribution. I do like the ideas of taking existing concepts from the Spring Ecosystem an applying them to Spring AI. I will drive deeper but what you are doing is in the right direction from a quick look.

I quite like the taxonomy/terminology in much of LangChain. Their definition of agent is

"Some applications will require not just a predetermined chain of calls to LLMs/other tools, but potentially an unknown chain that depends on the user's input. In these types of chains, there is a “agent” which has access to a suite of tools. Depending on the user input, the agent can then decide which, if any, of these tools to call."

So it is sort of the next step of chains. I see us leveraging Spring Integration's [IntegrationFlow](https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html) DSL to help take us to the next level of Agents. There are many classic EIP patterns that we can apply to AI usage cases. Same applies with use cases such as Retrieval Augmented Generation where we can create batch jobs to load embedding into a vector database.

BTW, the lowercase "i" in AiClient has been irking me a bit. Thoughts on that?

[Kirbstomper]

I'm all for capitalizing the i. AIClient does feel so much better than AiClient 😆

[markpollack]

There is a PR that will be reviewed related to this topic #16

[markpollack]

Man, I really apologize for not getting to this yet. I need to set aside some time for 'understanding functions' better. It will be merged!

[markpollack]

We are aiming for basic function support in 0.8.0. high level interactions, e..g Chat with conversational memory and agents etc, or high level APIs around function invocations will be in a later release. Not quite sure when. Just wanted to keep the thread alive, it has not been forgotten.

[mxab]

Hi,
just as another inspiration, I created a python function handler framework https://github.com/mxab/taifun where you can annotate function and it extracts the functions parameters as json schema and provides it to openai as functions to call.

Something similar in Spring would also work maybe nice. eg

record Location(float lat, float lon)
class AiController {
 @AiFunction("retrieves the location of the requester")
 public Location getLocation(){
 // get location
 }
 @AiFunction("returns the weather for a given location ")
 public String getWeather(@AiParam("the lat value....") Location location){
 // get weather
 }
}

AiFunctionsBean functions = generateFromAllAiControllers()

aiclient.generate("do i need an umbrella", functions)

[markpollack]

Function Calling support has been added. It don't think function calling itself is equivalent to an agent, an agent likely has a persistent memory and additional files, more akin to the open ai assistants api. Closing this issue now, but reopen if I've missed something.h