Can I add a function call to the response of a ChatCompletionsAgent inside my plugin? [Python]

[mkemmerz]

Hello, I have a bit of a special question.

I am using a ChatCompletionAgent (source) with multiple Plugins. One of the Plugins uses an AzureAIAgent (source) to execute things like code interpreter on the AI Foundry. This Plugin returns its result as string.

ChatCompletionsAgent -> My custom code Interpreter Plugin -> AzureAIAgent -> Code Interpreter Function Calls

Now the issue I have. Semantic Kernel responds with a FunctionCall to my Plugin. The AzureAIAgent itself also does Function Calls (e.g. the code interpreter). Is it possible to add these functions calls to the response of the ChatCompletionsAgent so that I can catch them via the on_intermediate_message?

Current response ChatCompletionsAgents returns with:
Semantic Kernel -> Function Call to my custom Code Interpreter Plugin

Wished ChatCompletionsAgents returns with:
Semantic Kernel -> Function Call to my custom Code Interpreter Plugin
                -> Function Call to code interpreter via my Code Interpreter Plugin

I know, technically they are different things so I would somehow need to get the correct Semantic Kernel object / function / whater is needed on Plugin level.

I already tried with with prompting so returning the complete RunSteps from the thread but Semantic Kernel ignored it basically. I also tried to use GroupChat but this didn't work out for us so I don't see it as a solution.

Happy about any hints :)

[mkemmerz]

I found a solution.

Short version:

• Pass ChatHistory object via kernel_arguments to the Plugin
• Add new Message with AuthorRole.Tool in the Plugin to the Chat History
• Parse the Chat History in the altered intermediate method to get the function call again on a higher level

kernel_arguments = KernelArguments(history=chat_history)
augmented_streaming_handler = partial(handle_streaming_intermediate_steps, chat_history=chat_history)
async for content in assistant(messages=chat_history, kernel_arguments=kernel_arguments, on_intermediate_message=augmented_streaming_handler):

@kernel_function(name="foo")
async def foo(self, arguments: KernelArguments):