GetChatMessageContentAsync update with infromation from SK - c#

[JP-droidic]

With a prompt that instructs an LLM to update the user on what it's doing, and GetChatMessageContentAsync we've manged to get updates like:

I will now fetch data from the database -> a call of SK. Sometimes, we do a lot of heavy lifting in the function, e.g.:

Validate user input, call LLM to generate SQL, run SQL, and possibly process the data fetched from the database—the call can last 2-5 seconds.

Is there a way to feed the LLM/and then the LLM to update the user via GetChatMessageContentAsync method, with custom messages from the SK like:

we are parsing your input
we are generating SQL
we are running the query
we are processing data

[dmytrostruk]

@JP-droidic If your operations like parsing an input, generating SQL, running the query are SK functions, then you could use IFunctionInvocationFilter to receive a call before each function is executed. When it happens, you can get an information about function, it's arguments, rendered prompt etc, so you can share this information with user.

Filter definition:

semantic-kernel/dotnet/samples/Concepts/Filtering/FunctionInvocationFiltering.cs

Lines 262 to 272 in de2f1bf

	private sealed class FirstFunctionFilter(ITestOutputHelper output) : IFunctionInvocationFilter
	{
	private readonly ITestOutputHelper _output = output;
	public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
	{
	this._output.WriteLine($"{nameof(FirstFunctionFilter)}.FunctionInvoking - {context.Function.PluginName}.{context.Function.Name}");
	await next(context);
	this._output.WriteLine($"{nameof(FirstFunctionFilter)}.FunctionInvoked - {context.Function.PluginName}.{context.Function.Name}");
	}
	}

Filter registration:

semantic-kernel/dotnet/samples/Concepts/Filtering/FunctionInvocationFiltering.cs

Line 28 in de2f1bf

builder.Services.AddSingleton<IFunctionInvocationFilter, FirstFunctionFilter>();

Please let us know if this could cover your scenario. Thanks!

[JP-droidic]

@dmytrostruk thank you. The issue for us is that all this happens in one SK function. So our function is called ExtractDataFromDatabase - and inside of that function, we do:

1. Parsing
2. Creating SQL
3. running SQL

We do not have 3 separate SK functions for this - it's just one.

The reason for this is we never managed to have a consistent function call chain from LLM with broken down calls

[dmytrostruk]

@JP-droidic Got it, thank you for more information! Could you please share more details how exactly you want to notify users about changes? What would be the desired outcome of such functionality? Thanks!

[JP-droidic]

Sure:

this is what we got from AI, before calling SK function

_Hey James., I'll start by extracting the data on deals closed in 2023 and their associated tags. Once I have that data, I'll calculate the ratio between the number of deals and the tags.

I'll get the necessary data now. This might take a moment, so please hang tight._

A few seconds later, we got:
Hey James., I have the data for deals closed in 2023. Here's the summary:

What we want to achieve is to update an LLM what is happening in the function, so ideally, to have it pushed by AI:

_Hey James., I'll start by extracting the data on deals closed in 2023 and their associated tags. Once I have that data, I'll calculate the ratio between the number of deals and the tags.

I'll get the necessary data now. This might take a moment, so please hang tight.

We are processing your question <- injected by us from SK

We are now running SQL, please keep tight we will get back to you shortly <- injected by us from SK

Hey James., I have the data for deals closed in 2023. Here's the summary:

[dmytrostruk]

@JP-droidic Thanks! So, if messages like We are processing your question, We are now running SQL, please keep tight we will get back to you shortly should be injected by you for the user, you don't exactly need to share these messages with LLM, right? You can send notifications directly to your user from SK function. If you are developing CLI tool, you can output these messages in Console, if it's web server, you can send notifications from SK functions using SignalR etc. Does that make sense?