Thoughts about Kernel Memory Core containing everything

[marcominerva]

Context / Scenario

I was thinking about pros and cons of having the Kernel Memory Core NuGet package contains all the extensions.

Question

Having all the extensions already included makes development and packaging a lot easier, but it has the drawback of adding a lot of unnecessary libraries to our custom deployments.

For example, even if I decide to use only Azure Storage and Azure AI Search, I still end up with DLLs like MongoDB.Bson.dll, MongoDB.Driver.dll, Npgsql.dll, NRedisStack.dll, Pgvector.dll, RabbitMQ.Client.dll and StackExchange.Redis.dll, and the list will grow every time new extensions will be created.

So, I would like to know why this decision has been made. In fact, actually the Core isn't a "Core" anymore, it is the whole Kernel Memory.

[dluc]

The primary goal of the recent changes was to reduce both development and management costs. All the individual nuget packages are secondary artifacts of the project, a consequence of the default assembly packaging and shipping process in .NET. While segregating code into separate assemblies is beneficial for scaling the project without escalating code complexity, maintaining 20+ packages imposes an unnecessary burden. Moreover, some developers might inadvertently depend on individual nugets out of context, leveraging the public interfaces and expecting additional features and support that may deviate from the project's objectives.

The observation about the "Core" encompassing all packages is valid. However, at this stage, we opted not to introduce another package like "Microsoft.KernelMemory" that would contain everything. We may reconsider this decision in the future, once other pressing tasks have been addressed.

As for the issue of deployment size, it can be mitigated by removing assemblies during the packaging/publishing step. While this might necessitate extra effort, having all the assemblies is helpful as it allows for dependency switching without the need to redeploy the solution. This flexibility is beneficial in many scenarios.

[marcominerva]

Thank you @dluc for the explanation 👍

[akordowski]

I would like to join the discussion regarding the question.

maintaining 20+ packages imposes an unnecessary burden

I agree, maintaining a lot of packages is not fun at all. But the SemanicKernel project also get it done with its 20 connector packages. The Microsoft.SemanticKernel.Core package is lean and is a core in it sense. I can add a specific connector when needed and don't have any overhead.

As I really like and appreciate the Kernel Memory project, but this is a downside for me, be forced to drag dependencies that I don't even need. I would prefer to have a lean Microsoft.KernelMemory.Core package.

and the list will grow every time new extensions will be created.

And as @marcominerva said the problem will grow with every new package.

So I would like to know, are there any considerations or priorities to fix this?

[dluc]

Here's the PR changing Core to include only core logic, introducing a new Microsoft.KernelMemory package that contains all extensions and clients.

#847

[akordowski]

That is great news, thank you very much! Looking forward to the lean core package 🥳

[dluc]

Update, starting from version 0.90

• Microsoft.KernelMemory is the new package containing all the dependencies
• Microsoft.KernelMemory.Core contains just core logic

[akordowski]

Thank you very much!