Adding multiple services of same type to the kernel (e.g. chat completion)

[akordowski]

This is a question just for understanding, as I couldn't find any clue in the documentaion.

The kernel builder offers extension methods for adding services of same type, like AddAzureOpenAIChatCompletion() and AddOpenAIChatCompletion() and much more. With kernel.GetRequiredService<IChatCompletionService>() the service can be retrieved.

Now I am wondering if:

• It is possible to add multiple services of the same type to the kernel (e.g. chat completion, text to image, ...)?
• Can this be done or should it be prevented?
• If it is possible how the kernel decides which one service to use?
• What is the best practice?

I hope anyone can give a explanation for better understaning. Thank you!

[dmytrostruk]

Hi @akordowski !

It is possible to add multiple services of the same type to the kernel (e.g. chat completion, text to image, ...)

Yes, you can add multiple services of the same type (e.g. multiple IChatCompletionService services).

Can this be done or should it be prevented?

This can be done, and it's especially useful when you want to switch to different chat completion endpoint in specific period of application runtime.

If it is possible how the kernel decides which one service to use?

We have IAIServiceSelector abstraction, which is responsible for service selection. If you don't specify any explicitly, default OrderedAIServiceSelector will be used. But it is possible to override default behavior and implement your own IAIServiceSelector based on your needs, like in this example.

What is the best practice?

I think it depends on your use-case. If you use one AI model and it works correctly for you, then one IChatCompletionService should be enough. If you want to handle scenarios like switching to another AI model in case of error, or use specific AI model to solve specific type of task, registering a couple of chat completion services may help you to handle this scenario correctly. Here is another example of retry logic achieved with multiple chat completion services and Filters: RetryWithFilters.cs.

I hope that helps!

[akordowski]

Thank you very much @dmytrostruk for the good explanation! Now I understand better how the things work and how they can be extended.