Complex Type interpretation in Function Calling

[ADDrian-P]

Hi,

While working with Function Calling in Semantic Kernel, I've encountered issues when the LLM attempts to map functions with complex types (objects that may contain other objects).

I have some questions regarding how the model gathers the necessary information to understand the structure and purpose of these parameters:

1. Does it rely solely on parameter and class names?
2. Does it consider standard C# XML documentation tags, such as /// <summary>...</summary>?
3. Does it only take into account the [Description("Description")] attributes on parameters and classes?

If XML documentation tags are not used, is there a recommended way to provide additional context to improve the LLM's understanding and avoid errors when invoking functions with nested structures?

Thanks in advance for your help!

[moonbox3]

Adding @SergeyMenshykh to help.

[SergeyMenshykh]

1. Does it rely solely on parameter and class names? 3. Does it only take into account the [Description("Description")] attributes on parameters and classes?

The class name, function names, and function parameter names are the main sources of method metadata shared with the model. The class name is used as the plugin name. Function names are provided to the AI as they are, unless an alternative name is specified via the KernelFunctionAttribute, i.e., [KernelFunction("my_function")], in which case the alternative name is provided to the model. It's also possible to provide additional context for each function using the DescriptionAttribute, i.e., [Description("Inserts a provided record into the vector store")]. Information about function parameters - such as name, type, and isRequired - is extracted from the method signatures and provided to the model. It is also possible to provide additional context for each parameter by annotating them with the DescriptionAttribute, i.e., [Description("Record ID")] string p1.

2. Does it consider standard C# XML documentation tags, such as /// <summary>...</summary>

No, XML comments are not considered.

If XML documentation tags are not used, is there a recommended way to provide additional context to improve the LLM's understanding and avoid errors when invoking functions with nested structures?

Please consider the following recommendations: https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/?pivots=programming-language-csharp#general-recommendations-for-authoring-plugins