Provide function return type schema

[ashahabov]

Provide function return type schema

Am I correct in understanding the above guideline states that the attributes Description and JsonPropertyName of the returned type LightModel will be ignored without using the statement kernel.AutoFunctionInvocationFilters.Add(new AddReturnTypeSchemaFilter())?

I tested a little on simple examples without this filter and it seems to work without it.

private sealed class AddReturnTypeSchemaFilter : IAutoFunctionInvocationFilter
{
    public async Task OnAutoFunctionInvocationAsync(AutoFunctionInvocationContext context, Func<AutoFunctionInvocationContext, Task> next)
    {
        await next(context); // Invoke the original function

        // Crete the result with the schema
        FunctionResultWithSchema resultWithSchema = new()
        {
            Value = context.Result.GetValue<object>(),                  // Get the original result
            Schema = context.Function.Metadata.ReturnParameter?.Schema  // Get the function return type schema
        };

        // Return the result with the schema instead of the original one
        context.Result = new FunctionResult(context.Result, resultWithSchema);
    }

    private sealed class FunctionResultWithSchema
    {
        public object? Value { get; set; }
        public KernelJsonSchema? Schema { get; set; }
    }
}

// Register the filter
Kernel kernel = new Kernel();
kernel.AutoFunctionInvocationFilters.Add(new AddReturnTypeSchemaFilter());

[Description("The state of the light")] // Equivalent to annotating the function with the [return: Description("The state of the light")] attribute
public class LightModel
{
    [JsonPropertyName("id")]
    [Description("The ID of the light")]
    public int Id { get; set; }

    [JsonPropertyName("name")]
    [Description("The name of the light")]
    public string? Name { get; set; }

    [JsonPropertyName("is_on")]
    [Description("Indicates whether the light is on")]
    public bool? IsOn { get; set; }

    [JsonPropertyName("brightness")]
    [Description("The brightness level of the light")]
    public Brightness? Brightness { get; set; }

    [JsonPropertyName("color")]
    [Description("The color of the light with a hex code (ensure you include the # symbol)")]
    public string? Color { get; set; }
}

[markwallace-microsoft]

@dmytrostruk can you respond to this one?

[dmytrostruk]

@ashahabov Thanks for your question.

When we share an information about functions with AI model, we include function name, description and parameters. For example, in OpenAI models there is no field in API that will accept an information about function return type:
https://platform.openai.com/docs/guides/function-calling#defining-functions

When function is invoked and we have a result, we serialize that result and return it back to AI model. During this serialization step, things like JsonPropertyName will be included to produce a string representation of function result, but not Description, since we are returning function result value and not function result schema.

In your case, it is working because you have a meaningful property names and JsonPropertyName attribute values. But if we change the example a little bit and return some property like Cost where Description will have some information about currency of that value, in this case that information will be ignored, unless you will use the technique described in the documentation.

The examples in this file better demonstrate the overall idea:
https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/FunctionCalling/FunctionCalling_ReturnMetadata.cs

semantic-kernel/dotnet/samples/Concepts/FunctionCalling/FunctionCalling_ReturnMetadata.cs

Lines 189 to 202 in c669f74

	public sealed class WeatherData
	{
	[Description("Temp (°C)")]
	public double Data1 { get; set; }
	[Description("Humidity (%)")]
	public double Data2 { get; set; }
	[Description("Dew point (°C)")]
	public double Data3 { get; set; }
	[Description("Wind speed (km/h)")]
	public double Data4 { get; set; }
	}

We will also update the documentation on Microsoft Learn with better code examples for this scenario, here is a task: #11202.

[ashahabov]

@dmytrostruk, thank you for the detailed explanation. I've tested it using the above test, and yes, the behavior is as you described.