How to "replicate" custom Copilot topics and flows

[stephane-liwerant]

From what I understand, to avoid having one agent with too much plugins and probably too much hallucination, topics could be managed by different agents persona. And the steps could be managed by specific functions for each agent, which will be triggered thanks to function calling.

My pain point here is that I understand that function calling will let AI (in my context Azure OpenAI GPT-4o) decide the plan, depending on the prompt. But the interest of flows in Copilot is that I can manage the differents steps, be explicit and consistent, even user-proxy ones.

How could I pre-define a plan, for some specific user intent, and don't let LLM get out of this plan? Probably, if the plan does not need any user input during the steps, it could be done programmatically. But what about plan with "user-proxy" step(s)?

[sophialagerkranspandey]

If you wanted to follow the patterns developed by AutoGen - you could create an "agent" that doesn't make requests to an LLM and instead makes requests to the user to provide information and include it as a step.
For your first question, to make it such that a LLM isn't left out of the plan, we recommend having this be a single function and calling on the steps to make sure the LLM can't deviate from the plan.

[stephane-liwerant]

For your first question, to make it such that a LLM isn't left out of the plan, we recommend having this be a single function and calling on the steps to make sure the LLM can't deviate from the plan.

Thanks @sophialagerkranspandey for your answer. Unfortunately, with Semantic Kernel, at least with the current version, I only see single function if all your steps don't need any human intervention, or at the end. How can I add human-input steps, and still manage the plan? Can I have a multi-agents plan (with the mentioned "user-proxy" agent) with a single function?

[madsbolaris]

We started discussing this internally in the team, and we believe there are a couple options you could take...

1. Always ask the user for information within the function – Within the function, you could make a request to your UI to ask the user for the additional information. If you are using HTTP requests, this is likely difficult because you would have to store state and rehydrate it once you got the user result. Because of this, we would recommend using something like SignalR / websockets to implement this approach.
2. Revert back to function calling loop if information is missing – If, however, you don't know whether you need to ask the user for information every time (e.g., some data is missing from a document), what you could do is have the AI run the function, and if the function requires more information (that isn't available), throw an exception that tells the AI that the function cannot proceed because it's missing some data. At that point, within the function calling loop, the AI will ask the user for the information and it can then save that information to the document. Afterwards, it can call the function again and this time, it will make it past that step.

[stephane-liwerant]

Thanks @matthewbolanos. I understand that technically we could reproduce a kind of workflow by coding it, but there is (for the moment at least) no graph or workflow management inside of Semantic Kernel. The current philosophy / strategy is to give assistant capabilities (former skills and now called plugins) and to let IA define the plan by himself, and ask for missing inputs.