Dynamic function calls with SK planners, e.g. Handlerbars planner

[bbence84]

I would like try the new Handlerbars based planner (or any other kind of planner, e.g. stepwise) with "dynamically" provided plugins. I have implemented a generic webservice in my application which returns a the function call metadata in the old OpenAI format (but it could also be the new tools format). So it essentially would return a JSON array of OpenAI function call definitions. Example:

functions= [  
    {
        "name": "get_db_table_fields",
        "description": "Returns the list of DB table fields for the specified table name",
        "parameters": {
            "type": "object",
            "properties": {
                "table_name": {
                    "type": "string",
                    "description": "The name of the DB table"
                }
            },
            "required": ["table_name"]
        }
    }
	...
	OTHER FUNCTIONS
	...
] 

I have another generic webservice that I can call. This generic receives the name of the function, all the parameters of the function (there can be 0 or many, with different names.

I already have a POC in Python with Microsoft Autogen, and it works fine. The following is a part of the Python code that one hand gets the metadata and creates a dict that can be passed to the Autogen Agent so that when a function call is picked by the LLM model, it calls the function. This is done with a lambda proxy function. This proxy function then calls this generic backend service I mentioned, see below:

def get_backend_function_call_meta():   
    # returns the OpenAI function call metadata JSON array of the function calls from the backend 
    backend_functions = function_call_backend.get_functions_metadata_for_class()
    # returns a dictionary of the function calls from the backend, mapped to the function to be called via the proxy lambda function
    backend_func_dict = {funct['name']: (lambda x: lambda **data: func_call_proxy(x, **data))(funct['name']) for funct in backend_functions}
    return backend_functions, backend_func_dict

def func_call_proxy(function_name, **data):
    # calls the function in the backend
    return function_call_backend.call_function_proxy(function_name, **data)

All these backend functions return a JSON, e.g. something like the following:
{"TABLE_NAME":"CUSTOMERS","FIELD_LIST":["CUSTOMER_ID","CUSTOMER_NAME","CUSTOMER_ADDR","CUSTOMER_EMAIL"]}

I thought I would provide a Prompt template (Semantic Function) that could format the output of given functions. Not sure if Semantic Functions are appropriate for this though, but the Planner could ideally include a step to call the backend function first, then use the Prompt template to return a natural language description of the backend function result JSON.

Now the question is how can I implement / port my Python based Autogen implementation to SK and C# (I am not very well versed in C# to be honest). So essentially it boils down to this:

1. How can I pass these dynamically retrieved functions from the backend to SK planners, e.g. HandlebarsPlanner, so it can include them?
2. Can I also add additional Semantic Functions (Prompt templates) the Planner so that it can use these "formatters" to return the natural language response?

[bbence84]

I managed to solve this, kind of. :) I essentially created plugins for the functions, as well as for the planner and with some additional system prompt tuning, I was able to get it to work.
The dynamic Plugin creation part is still pending, but for now with hardcoded methods, it's fine for a POC.