Auto function calling in unwanted loop - ChatCompletionAgent

[Mantis748]

I'm attempting to build out my first SK agent to mimic one I have working using OAI's assistant api. I am following this:
https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/examples/example-chat-agent?pivots=programming-language-csharp

This agent will:

• accept user input requesting analysis of log data
• call a plugin that generates sql for the log db (this is working well in terms of generating against the schema)
• use the generated sql to execute against a DB and generate CSV
• analyse the CSV. Initially, I'm letting the LLM do simple analysis against it but eventually will be doing a codegen step that will execute the query

The issue is this: the sql gen plugin is executed again and again, even if the output is correct. I wrote a simple invocation filter to intercept the calls. I see no errors, and I do see the sql being added to the chat history.

So what might I be missing? The history has the generated sql, but for some reason SK and/or the LLM don't seem to recognize it and pass it on to the next plugin.

Calling code below:

public async Task Run()
{
    var kernel = Program.BuildKernel();

    ChatCompletionAgent agent =
        new()
        {
            Name = "LogAnalysisAgent",
            Kernel = kernel,
            Instructions = """
                You are an AI data analysis assistant for the insurance industry. You can answer questions about insurance concepts by searching the
                documentation, and you can also analyze the provided log database for insights
            
                You have access to the following tools:
                - WikiSearch: for finding relevant internal documentation
                - SqlGen: for generating SQL queries based on the user’s question
                - SqlToCsv: for executing a SQL query and returning results as csv
            
                ## Instructions:
                - If the user asks a question about internal documentation, use the WikiSearch plugin.
                - If the user asks a question that requires analyzing data, use SqlGen to generate the SQL.
                - Sql should ONLY be generated using the provided plugin. DO NOT make any DB schema assumptions or hypotheticals.
                - Once you have the generated SQL then convert to result to csv. Locate the sql with a block marked ```sql
                - **Do not generate another SQL query if one already exists.**
                - Use the result from the csv to answer the user's question.
                - Always return a clear and concise response to the user, based on either the documentation or the SQL results.

                The current date & time is {{$now}}
                The tenant id for the sql query is {{$tenant}}

            """,
            Arguments = new KernelArguments(new OpenAIPromptExecutionSettings()
            {
                FunctionChoiceBehavior = FunctionChoiceBehavior.Auto(),
                //ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
            })
        };

    ChatHistoryAgentThread agentThread = new();
    bool isComplete = false;
    do
    {
        Console.WriteLine();
        Console.Write("> ");
        string input = Console.ReadLine();
        if (string.IsNullOrWhiteSpace(input))
        {
            continue;
        }
        if (input.Trim().Equals("exit", StringComparison.OrdinalIgnoreCase))
        {   
            isComplete = true;
            break;
        }

        var message = new ChatMessageContent(AuthorRole.User, input);

        // so invoke agent here after user input
        // seems that agent will consider input, then choose proper tools to run
        //      does it keep tool output in some sort of memory?
        //      to test: can it invoke multiple tools and string them together?
        // 

        Console.WriteLine();

        DateTime now = DateTime.Now;
        KernelArguments arguments =
            new()
            {
                { "now", $"{now.ToShortDateString()} {now.ToShortTimeString()}" },
                { "tenant", $"8" }
            };
        await foreach (ChatMessageContent response in agent.InvokeAsync(message, agentThread, options: new() { KernelArguments = arguments }))
        {
            if (response is OpenAIChatMessageContent)
            {
                OpenAIChatMessageContent oaiResponse = (OpenAIChatMessageContent)response;
                var wut = oaiResponse.Items;
            }
            // Display response.
            Console.WriteLine($"{response.Content}");
        }
    } while (!isComplete);
}

[sophialagerkranspandey]

Tagging @SergeyMenshykh

[SergeyMenshykh]

Hi @Mantis748, may I ask you to provide two plugins that would reproduce the issue: one that generates the SQL (it's fine to return hardcoded SQL as long as it reproduces the issue) and the other that expects the SQL as input? Both can be fake/stubbed ones as long as they help to reproduce the issue on my side.

[SergeyMenshykh]

Closing for now. @Mantis748 feel free to reopen it when you have the plugins so we can use them to repro the issue.