Tool grouping

[MackinnonBuck]

Summary

This PR demonstrates a prototype middleware that allows grouping related tools. Groups are hidden from the inner chat client by default, but a well-known "expand" function can be called to expand one group at a time. By reducing the number of tools available to the model at a time, its ability to select the correct tool improves.

Following is an overview of the changes in this PR:

• Add ToolGroupingChatClient, which provides tool grouping and expansion functionality.
• Introduce AIToolGroup to model stable, named batches of tools.
• Add integration tests validating the feasibility of approach.

Usage example

using Microsoft.Extensions.AI;

var travelUtilities = AIToolGroup.Create(
    "TravelUtilities",
    "Travel planning helpers that generate itinerary tokens.",
    [generateItinerary, summarizePacking]);

var financeUtilities = AIToolGroup.Create(
    name: "FinanceUtilities",
    description: "Budgeting helpers.",
    tools: [budgetTool]));

var options = new ChatOptions
{
    Tools = [travelUtilities, financeUtilities]
};

IChatClient client = baseClient
    .AsBuilder()
    .UseToolGrouping(/* ... */)
    .UseToolReduction(/* ... */) // Optionally further reduce the set of available tools
    .UseFunctionInvocation()
    .Build();

var response = await client.GetResponseAsync("Plan a weekend getaway to Lisbon and include a budget tip.");
// ...

Implementation notes

This middleware contains a lot of the same complexities present in FunctionInvokingChatClient: cloning options, honoring ConversationId, aggregating Usage across inner calls, and replaying tool-result messages into the history. The expansion loop is deliberately limited to one active group per top-level request to prevent degradation of tool selection as the number of expanded groups increases.

Microsoft Reviewers: Open in CodeFlow

[ericstj]

This is very clever, I had a couple small questions on behavior.

[ericstj]

So we only group tools known at initialization? What about those added per call to GetResponse? I could imagine some sort of callback to assign a group. Or we could let folks annotate a tool with a group and store that in AdditionalProperties.

[ericstj]

Another thought is auto-grouping based on similarity of descriptions / embedding.

[MackinnonBuck]

That's definitely something to consider. We could have a way to annotate a tool with a "category", similar to how tool descriptions are defined. Then chat clients like ToolGroupingChatClient can utilize the category to perform grouping on a per-response basis. This keeps the tool configuration completely in ChatOptions rather than having it be split between ChatOptions and other middleware.

Or maybe we could change the AIToolGroup type to extend AITool. That way you could put tool groups directly in the ChatOptions. You could even make AIToolGroup unsealed and have a virtual GetToolsAsync() method that can be overridden if you want to dynamically expose different sets of tools based on some other logic (this could be used to implement the "get more tools" function you mentioned in your other comment). I'll think about this a bit more.

[MackinnonBuck]

Some notes:

• We probably need to update FunctionInvokingChatClient to handle tool groups (likely just expanding all the tools in all groups).
• Instead of ToolGroupingChatClient, we could consider calling it ToolCollapsingChatClient or ToolExpandingChatClient.
• We could also consider having some kind of automatic grouping mechanism so that the developer gets the benefits of tool groups without having to define them manually. This could be done as a follow-up.