Tell us what you think about the Semantic Kernel v1 proposal

[madsbolaris]

Have thoughts about the proposal for v1 for Semantic Kernel? Any scenarios that you think won't be supported with these changes? Please share them here so we can make sure we incorporate them.

This discussion has been closed now that the blog post for the v1 proposal has gone live. If you have additional feedback, we still want to hear from you. please share them on our new discussion board here: microsoft/semantic-kernel#3358

[lemillermicrosoft]

var results = await function.InvokeAsync(kernel, variables);

or

var results = await kernel.RunAsync(function, variables);

Which should I do and when?

Also, what is the type of results in each of these. Will SKContext be part of the result object?

[madsbolaris]

IMO, these two call patterns will likely be identical since both should return a FunctionResult. Instead of getting an SKContext in the result (since our goal is to remove SKContext by v1.0.0), you'll simply get a Dictionary<string, object> within the FunctionResult with all of the final returned variables.

[Eddie-Hartman]

If they are identical, is there a reason to keep both? Seems like it would become a source of confusion in the future.

[madsbolaris]

After talking to the team, it looks like there will be one subtle difference between the two method signatures.

• kernel.RunAsync() can also take in optional parameters for kernel-level execution settings (e.g., pre/post hooks, execution overrides, etc.)
• function.InvokeAsync() meanwhile will always get its execution settings from the kernel object. The main reason why this function will persist is because the kernel will need something to call internally within the kernel.RunAsync() method.

Documentation should be written to recommend that the kernel.runAsync() method be used. Any thoughts on the above?

[runceel]

I think this constraint is a constraint of HandlebarsDotNet, but if HandlebarsPromptTemplateEngine does not support RenderAsync, I think there will be performance issues when used on the server side, and deadlock issues when used in WPF and so on. Do you plan to address this issue? Or am I missing something?

[madsbolaris]

Yes, this is currently a constraint of Handlebars. In the final solution we'll make the Render function RenderAsync for IPromptTemplateEngine (or it's equivalent). As a developer, you'll then have a choice between Handlebars (which behind the scenes doesn't support async), f-strings, our existing template language, and jinja2 (for Python only)

[runceel]

In my opinion, the constraint is a serious limitation for Web and Windows app developers. Therefore, after releasing v1, it would be better if the constraint were documented in the SK document.

[eavanvalkenburg]

I like the setup, I was also looking at how to serialize the chat setup in a single structure (in python), with or without history (this should be split anyway). Otherwise the overall thinking is good and partly already there in python (ai services and skills are already separate objects in python), one thing I'm not so sure about is the dumping of the context, I think there are things to improve, but doing everything with dicts in python becomes very ugly (lots of if 'field in result: do this`) versus just being able to count on i.e. variables being present.

[eavanvalkenburg]

You would also need it to parse results (assuming the result is a dict as well, either new or updated from what you passed in) so then you would deal with it there as well. So generally in python you could just define it is a dict[str, Any] and then everything works, but this might lead to some headaches, but I am not sure we can get away from that, since the run_async on the kernel needs a type and that therefore will almost have to be Any to allow all sorts of things to be passed.

[eavanvalkenburg]

actually that is a big assumption, will the result always be a updated dict of the variables that were passed in or can that also just be the string result?

[madsbolaris]

In the .Net version, the results of functions will always be typed. So for simple things like text completion semantic functions, the result will just be a string. More complex functions (e.g., an API call), however, may return a complex object.

The thing that invoked the function should then be aware of how to handle the type that was returned back. You can sort of see this happen today with primitive types in the FunctionResult.GetResult() function. What I'm not sure of is how that same logic can appear in Python.

[eavanvalkenburg]

That can be done. Python is not strongly typed anyway, I do think it is easier to wrap all results in a dict with one of more keys then getting back single objects or tuples or dicts... And the only real type hint would be at the function level while I believe the idea is that most calls do go through kernel.run_async which will be cumbersome to type with anything other then Any.

[eavanvalkenburg]

@matthewbolanos I was thinking about this some more and with the notion of post-hooks then I think dicts are really the way to go, the way I see it is: every function gives back a dict, with 1 required field response_object and 1 or more "content" fields, for semantic functions that might also be a default name like result or something similar, and for native functions that should be available to change based on the developer wishes (or maybe it's at least result and optionally other fields). The post-hooks come into play to then allow the kernel to take the result and/or response_object and do manipulation on the whole thing, so you can easily create a post-hook that takes the dict and just returns the content of the result field, while also allowing easily doing things with token usage, content filters, multiple responses, etc. I think this gives both plugin developers and app developers the ease of use of knowing where the response is and being able to quickly handle it, but at once giving the full features of the LLM to them (through response_object) and lots of flexibility in handling different responses with hooks, for instance for streaming results and other more complex setups, SK can then even supply some standard post-hooks/callbacks to make common scenario's easier, like streaming, multiple responses, tokens and contentfilter. In this approach the post-hook take the result dict as argument and return either None (for a side-effect) or they overwrite the result object, with a updated dict, or replacing it entirely with a different type (kernel.run_async will then have to have Any as type in Python)

[KSemenenko]

I love idea to use Memory Kernal as Plugin.

[madsbolaris]

This discussion has been closed now that the blog post for the v1 proposal has gone live. If you have additional feedback, we still want to hear from you. please share them on our new discussion board here: microsoft/semantic-kernel#3358