Mark tool parameter as ignored

[nmccann]

With the recent introduction of kwargs flowing through to tools, it is now possible to have an ai_function that does this:

@ai_function
def search_tool(query: str, **kwargs: Any) -> str:
   deterministic_filter = kwargs.get("deterministic_filter")

   return f"Performed query {query} with filter {deterministic_filter}"

And call it like so:

# Set up agent that uses the search_tool

result = await agent.run("How fast is a cheetah?", deterministic_filter="animals")

With this, the agent's LLM will see that search_tool expects a query parameter that should be a str, and it will populate that parameter. The deterministic_filter will end up propagating through to the kwargs and ensure that the filter is always the same, and not subject to the whims of the LLM.

This kwargs approach is nice, but has downsides. For one, additional work is needed to extract the value from kwargs before it can be used (and potentially requires casting it to the desired type). Without looking at the implementation, it's unclear what kwargs are applicable for the search_tool, or what type they should be.

An alternative would be:

@ai_function
def search_tool(query: str, deterministic_filter: str, **kwargs: Any) -> str:
   return f"Performed query {query} with filter {deterministic_filter}"

In my testing the end result is the same, because the value from kwargs takes priority over the LLM's generated value. However this isn't ideal as the LLM potentially wastes tokens on parameters it shouldn't worry about, and creates ambiguity about whether the value of the parameter is deterministic or not.

My idea would be to leverage some of Pydantic's capabilities (maybe combining SkipJSONSchema with Annotated) to denote a parameter as ignored for the purposes of the tool's representation to the LLM.

I thought SemanticKernel had something similar, but the closest thing I was able to find with some quick searching was this which doesn't seem to have been implemented.

Edit: In my testing, the second example actually uses the LLM's generated value if you omit **kwargs: Any from the function definition. I'm not sure why that would be the case.

[moonbox3]

Hi @nmccann, you are correct, we have a way in SK Python to exclude certain function arguments from the LLM: https://github.com/microsoft/semantic-kernel/blob/dc7c1c048488a8b611b5382d0d7efe05608a9fa0/python/semantic_kernel/functions/kernel_function_decorator.py#L34C89-L36C68.

@eavanvalkenburg are we tracking anything in MAF to support similar functionality like we had in SK?

param: Annotated[str, {"include_in_function_choices": False}]

As proposed by @nmccann, it'd be:

param: Annotated[str, SkipJsonSchema]

[eavanvalkenburg]

That's a good question @nmccann there are indeed already three ways to do this, one is through kwargs (although as you point out that has some downsides and I'm not sure if we want to keep doing that, because the risk is that kwargs are needed for different things, for instance ContextProviders).

The second is to wrap a function in a class and then use class attributes/properties, so you could do this:

class FunctionContext:
    def __init__(self, filter: str):
        self.filter = filter

    @ai_function
    async def query(self, query: Annotated[str, "The query to execute."]) -> str:
         return f"Performed query {query} with filter {self.filter}"

functions = FunctionContext(filter=....)

agent = ChatAgent(..., tools=function.query)

# optionally, update the filter:
# functions.filter = "..."

agent.run("How fast is a cheetah?")

This is obviously slighty less dynamic, but makes sense.

The third is to use FunctionMiddleware.

The latter two are my preference because they (potentially) operate at a per-function level, rather then the blunt passing through of kwargs that is done with the first method, which in general is my concern with that approach, since it might expose things to other functions, which is not a big issue, but could be once you get more complex setups with other teams building tools, that you consume.

So let me know what you think Noah!

[nmccann]

Thank you for the reply, in my case I do need the dynamism that the kwargs approach provides, as I'm expecting a parameter that could change per-run (in my case, user calls an API and the request body contains a boolean flag indicating if the tool should behave in one way or another).

Originally I was using two different versions of the workflow (one with a tool configured one way, one with the tool configured the other way), that seems like it would work with the wrap a function in a class approach, but I wanted to avoid the mental overhead of having these two similar but slightly different instances of the same workflow, so leveraged the kwargs approach to parameterize the function.

I haven't tried the middleware approach yet, so if that could provide similar per-run dynamism, that would be a viable alternative.