Qdrant with Aspire times out

[JohnGalt1717]

Context / Scenario

If you spin up Qdrant using .net Aspire like this:

var vectorDb = builder
    .AddQdrant(ResourceConstants.VECTOR_DB,) //Passing ports also doesn't work
    .WithDataVolume();

Ensuring that your project has a WithReference, and then spin up Kernel Memory by creating a qdrantConfig from the connection string (which is verified to have valid http and apikey) and then passing .WithQdrantMemoryDb(qdrantConfig)

What happened?

When you try and import a document you get a timeout within the qdrant memory db code.

A qdrant client works just fine.

It should just work.

It would be really nice if Kernel Memory wrapped a proper QdrantClient instead of using direct Http requests because then we could use the service provider to get a qdrant client and pass it into WithQdrantMemoryDb and know it works and everything would integrate nicely with Aspire.

Failing that, it would be nice if there were samples on how to integrate with Aspire with Kernel Memory for everything.

Importance

I cannot use Kernel Memory

Platform, Language, Versions

Windows, C#, .NET 9.0 RTM with Aspire 9.0.0

Relevant log output

at System.Net.Http.HttpConnection.<SendAsync>d__56.MoveNext()
   at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__50.MoveNext()
   at System.Net.Http.Metrics.MetricsHandler.<SendAsyncWithMetrics>d__5.MoveNext()
   at System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10.MoveNext()
   at System.Net.Http.RedirectHandler.<SendAsync>d__4.MoveNext()
   at System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d.MoveNext()
   at Microsoft.KernelMemory.MemoryDb.Qdrant.Client.QdrantClient`1.<ExecuteHttpRequestAsync>d__14.MoveNext()
   at Microsoft.KernelMemory.MemoryDb.Qdrant.Client.QdrantClient`1.<CreateCollectionAsync>d__3.MoveNext()
   at Microsoft.KernelMemory.Handlers.SaveRecordsHandler.<CreateIndexOnceAsync>d__19.MoveNext()
   at Microsoft.KernelMemory.Handlers.SaveRecordsHandler.<InvokeAsync>d__13.MoveNext()
   at Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator.<RunPipelineAsync>d__11.MoveNext()
   at Microsoft.KernelMemory.Pipeline.BaseOrchestrator.<ImportDocumentAsync>d__23.MoveNext()
   at Microsoft.KernelMemory.MemoryServerless.<ImportTextAsync>d__11.MoveNext()
   at Microsoft.KernelMemory.MemoryServerless.<ImportTextAsync>d__11.MoveNext()

[JohnGalt1717]

PS: I suspect this is because of how the http client is being created inside of the Qdrant client. It's breaking on the proxy stuff in aspire.

[dluc]

could you provide the full aspire builder code? it might be about the hostname to be set in the env var

[JohnGalt1717]

builder
    .AddProject<ArticleAnalysis>(ToolConstants.ARTICLE_ANALYSIS)
    .WithReference(vectorDb)
    .WithReference(ai)
    .WithReference(aiModel)
    .WithReference(aiEmbeddingModel)
    .WaitFor(aiModel)
    .WaitFor(aiEmbeddingModel);

I assume this is what you're referring to?

And the endpoint that is passed in is: http://localhost:16334 which is the grpc port set by aspire.

[dluc]

are you loading KM as a docker image or as a project?

KM also needs Qdrant config, how are you passing those settings?

The env var for Qdrant endpoint is KernelMemory__Services__Qdrant__Endpoint, are you setting it?

[JohnGalt1717]

I'm using it as a project, not a docker image. I'm using the nuget packages.

[JohnGalt1717]

Update: If i hard code the http URL it works. The problem is that because it isn't using the qdrant client (GRPC) and is httping everything (slower), it can never work with what is passed by Aspire without hacking. (of which you have no way in production versus dev to know what the ports are)

[dluc]

Without access to code it's too hard to say, my guess is that some required configuration is not being passed to KM, so KM doesn't know which port to use when connecting to Qdrant, or simply might be using the default settings

[JohnGalt1717]

Without access to code it's too hard to say, my guess is that some required configuration is not being passed to KM, so KM doesn't know which port to use when connecting to Qdrant, or simply might be using the default settings

See my comment that we posted at the same time.

[dluc]

Update: If i hard code the http URL it works. The problem is that because it isn't using the qdrant client (GRPC) and is httping everything (slower), it can never work with what is passed by Aspire without hacking. (of which you have no way in production versus dev to know what the ports are)

in prod it's probably easier because aspire proxy is not present. You could try disabling the proxy locally too, and just using the default ports, ie 6333 for qdrant

[JohnGalt1717]

Update: If i hard code the http URL it works. The problem is that because it isn't using the qdrant client (GRPC) and is httping everything (slower), it can never work with what is passed by Aspire without hacking. (of which you have no way in production versus dev to know what the ports are)

in prod it's probably easier because aspire proxy is not present. You could try disabling the proxy locally too, and just using the default ports, ie 6333 for qdrant

You can't do that without creating duplicate mappings because the default ones will also be created and then the connection string will still be the ones automatically created by aspire in the connection string information. The connection string info from aspire correctly only has the GRPC endpoint, not the http endpoint since Qdrant prefers GRPC and that's the default for the qdrantClient which KM isn't using.

So basically the only way right now to use KM with Aspire is to just defeat Aspire entirely and pretend it doesn't exist and hard code everything.

KM Qdrant should be using the QdrantClient under the hood.

(and yes, Aspire is a HUGE use case for KM and should be heavily tested for all KM connectors)

[dluc]

Aspire like Kubernetes and others is an orchestrator, and shouldn't require changes to the code being hosted specifically for it. Otherwise, we'd have to change the code for 10 different orchestration engines :-)

I believe the issue here is simpler, e.g. what Aspire assumes about connection strings is not how every app works. For instance, KM doesn't use a "Connection String".

Anyway, I'll publish an Aspire example as soon as I find the time.

[JohnGalt1717]

Here's my KM setup:

builder.Services.AddKeyedSingleton(
    AIConstants.LOCAL_KEY,
    (sp, _) =>
    {
        var qdrantClient = sp.GetRequiredService<QdrantClient>();
        var qdrantConnectionString =
            builder.Configuration.GetConnectionString(ResourceConstants.VECTOR_DB)
            ?? throw new InvalidOperationException(
                "Vector Db connection string is invalid"
            );

        var qdrantConfig = VectorDbExtensions.ParseConnectionString(qdrantConnectionString);

        var memoryBuilder = new KernelMemoryBuilder()
            .WithQdrantMemoryDb("http://localhost:16333", qdrantConfig.APIKey) // <--- I'm hard coding this right now to the http port to make it work. It should just take qdrantConfig
            .WithOllamaTextEmbeddingGeneration(ollamaConfig, new GPT4oTokenizer())
            .With(
                new TextPartitioningOptions
                {
                    MaxTokensPerLine = 150,
                    MaxTokensPerParagraph = 500,
                    OverlappingTokens = 30,
                }
            )
            .WithOllamaTextGeneration(ollamaConfig, new GPT4oTokenizer())
            .With(
                new TextGenerationOptions
                {
                    Temperature = 0.0,
                    FrequencyPenalty = 1.4,
                    ResultsPerPrompt = 1,
                }
            );
        return memoryBuilder.Build();
    }
);

.WithQdrantMemoryDb("http://localhost:16333", qdrantConfig.APIKey) should be able to take a qdrantClient from sp.GetRequiredService() which was previously injected with builder.AddQdrantClient("aspireresourceid")

Then this would be aspire friendly and work. Any other clients that KM uses that uses 3rd party clients should work the same way.

[JohnGalt1717]

Aspire like Kubernetes is an orchestrator, and shouldn't require do change the code being hosted specifically for it. Otherwise we'd had to change the code for 10 different orchestration engines :-)

I believe the issue here is simpler, e.g. what Aspire assumes about connection strings is not how every app works. For instance, KM doesn't use a "Connection String".

Anyway, I'll publish an Aspire example as soon as I find the time.

The real issue is that even though I split the connection string into the endpoint and apikey and provide KM with that, the endpoint provided by aspire is the GRPC endpoint which KM doesn't use because KM is using a custom http qdrant client instead of the Qdrant provided client.

Here's the code that parses the connection string that the Aspire.Hosting.Qdrant library injects:

    internal static QdrantConfig ParseConnectionString(string? connectionString)
    {
        string endpoint;
        string? key = null;
        if (Uri.TryCreate(connectionString, UriKind.Absolute, out var uri))
            endpoint = uri.ToString();
        else
        {
            var connectionBuilder = new DbConnectionStringBuilder
            {
                ConnectionString = connectionString,
            };

            if (
                connectionBuilder.ContainsKey("endpoint")
                && Uri.TryCreate(
                    connectionBuilder["endpoint"].ToString(),
                    UriKind.Absolute,
                    out var serviceUri
                )
            )
            {
                endpoint = serviceUri.ToString();
            }
            else
            {
                throw new InvalidOperationException(
                    "endpoint is not part of the connection string."
                );
            }

            if (connectionBuilder.ContainsKey("key"))
                key = connectionBuilder["key"].ToString();
        }

        return new QdrantConfig { Endpoint = endpoint, APIKey = key ?? string.Empty };
    }

But I shouldn't have to do this, because KM should be using QdrantClient from their nuget package, not a custom rolled http client.

[dluc]

by the way, the qdrant client didn't exist when we started 3 years ago :-) We'll take a look, thanks

[JohnGalt1717]

Makes sense! Thanks! I appreciate it!

[dluc]

Here's a working example:

var builder = DistributedApplication.CreateBuilder(args);

var qdrant = builder.AddContainer("qdrant", "qdrant/qdrant")
    .WithHttpEndpoint(targetPort: 6333)
    .PublishAsContainer();

var qdrantEndpoint = qdrant.GetEndpoint("http");

builder.AddProject<Projects.Service>("kernel-memory")
    .WaitFor(qdrant)
    .WithEnvironment("KernelMemory__TextGeneratorType", "AzureOpenAIText")
    .WithEnvironment("KernelMemory__DataIngestion__EmbeddingGeneratorTypes__0", "AzureOpenAIEmbedding")
    .WithEnvironment("KernelMemory__DataIngestion__MemoryDbTypes__0", "Qdrant")
    .WithEnvironment("KernelMemory__Retrieval__EmbeddingGeneratorType", "AzureOpenAIEmbedding")
    .WithEnvironment("KernelMemory__Retrieval__MemoryDbType", "Qdrant")
    .WithEnvironment("KernelMemory__Services__Qdrant__Endpoint", qdrantEndpoint)
    .WithEnvironment("KernelMemory__Services__Qdrant__APIKey", "")
    .WithEnvironment("KernelMemory__Services__AzureOpenAIText__Endpoint", "https://....openai.azure.com/")
    .WithEnvironment("KernelMemory__Services__AzureOpenAIText__Deployment", "gpt-4o")
    .WithEnvironment("KernelMemory__Services__AzureOpenAIEmbedding__Endpoint", "https://....openai.azure.com/")
    .WithEnvironment("KernelMemory__Services__AzureOpenAIEmbedding__Deployment", "text-embedding-ada-002");

builder.Build().Run();

[JohnGalt1717]

This is a hack around it by avoiding using the Aspire.Hosting.Qdrant package and the client. It isn't really a working example because it completely defeats the purpose of what Aspire.Hosting.Qdrant does.

[dluc]

KM is designed to work with multiple storage solutions, regardless of how Aspire.Hosting.Qdrant is set up. As with other dependencies, the idea is simply to start KM and everything it relies on. There’s no "hack" involved in this approach.

[JohnGalt1717]

There is a hack because Aspire isn't just "docker compose in c#". It's purpose is to be able to have developers work as if they were in production AND generate everything necessary to push to production.

The official Aspire.Hosting.Qdrant package does that. Your code does not. It hacks in the use of the legacy REST API for sake of a connection string that doesn't include the URL to the legacy REST API that you're using and thus defeats everything else.

The only viable solution for this to not be a hack is to use the actually supported by Qdrant, client, since KM is also using the legacy REST interface which is also significantly slower than what the official client does with GRPC.

[JohnGalt1717]

PS: I expect this is also true of ALL other 3rd party dependencies like Ollama etc.

[dluc]

Just to clarify:

• KM uses a .NET client that predates Qdrant's official .NET client and remains backward compatible for about three years. In contrast, the newer client only supports gRPC and isn’t backward compatible.
• KM Qdrant's configuration hinges on just two parameters: Endpoint and API Key. If you need to integrate with Aspire.Hosting.Qdrant, extract those parameters from the connection string and supply them to KM, see below.
• Like many other applications, KM wasn’t built for Aspire, Kubernetes, or Docker Compose, even better it runs wherever you prefer, needing only a configuration file or equivalent environment variables. You can choose a docker image for AMD or ARM arch, you can run the service from code, you can choose a serverless embedded component, and also switch to a KM web client at any point.
• KM is open-source and MIT licensed, so you're free to adapt, improve, and extend it as you see fit. Everyone has their own approach, and that’s fine. At the end of the day, use whatever works best for you.

How to use Aspire.Hosting.Qdrant:

// Note: this works only for local development. In Prod values must be set manually
static string GetConnectionStringPart(IResourceWithConnectionString resource, string pattern)
{
    string? value = resource.ConnectionStringExpression.GetValueAsync(default).Result;
    return value == null ? string.Empty : Regex.Match(value, pattern).Groups.Cast<Group>().Skip(1).FirstOrDefault()?.Value ?? string.Empty;
}

var qdrant = builder.AddQdrant("qdrant");

builder
    .AddProject<Service>("kernel-memory")
    // ...
    .WithReference(qdrant)
    .WithEnvironment("KernelMemory__DataIngestion__MemoryDbTypes__0", "Qdrant")
    .WithEnvironment("KernelMemory__Retrieval__MemoryDbType", "Qdrant")
    .WithEnvironment("KernelMemory__Services__Qdrant__Endpoint",
        () => GetConnectionStringPart(qdrant.Resource, "Endpoint=([^;]+)"))
    .WithEnvironment("KernelMemory__Services__Qdrant__APIKey",
        () => GetConnectionStringPart(qdrant.Resource, "Key=([^;]+)"));

Hope this helps. Have a good one.