Looking to reuse the clients and minimize the socket creation while respecting the DNS changes at the same time.

[akshay-zz]

Currently, I'm caching the HTTP clients on the basis of the base URL/named client and managing them through a factory class(singleton). While injecting them with IHttpClientFactory to the factory class. But when I went through the articles of Stevej Gordon.
https://www.stevejgordon.co.uk/httpclient-connection-pooling-in-dotnet-core
https://www.stevejgordon.co.uk/introduction-to-httpclientfactory-aspnetcore

I found out that IHttpClientFactory is managing the HttpClientHandler and reusing it. Also, we can configure the PooledConnectionLifetime & PooledConnectionIdleTimeout. That will solve the issue we use to get while treating HTTP clients as a singleton.

So my question do I need to cache my HTTP clients? Or I'm unnecessarily managing and caching them. And can I just go ahead and inject the IHttpClientFactory client into my scope injections?

Also, I got these guidelines
https://docs.microsoft.com/en-us/dotnet/fundamentals/networking/http/httpclient-guidelines

Use a static or singleton HttpClient with PooledConnectionLifetime set to the desired interval, such as two minutes, depending on expected DNS changes. This solves both the socket exhaustion and DNS changes problems without adding the overhead of IHttpClientFactory. If you need to be able to mock your handler, you can register it separately.

But after reading so many articles and theards I'm confused about what should I do and what should I not. I saw you are quite involved in quite of threads. So more I'm looking for advice what the best way to use IHttpClientFactory if I'm looking to reuse the clients and minimize the socket creation while respecting the DNS changes at the same time.

We were having some conversation with @CarnaViire

[akshay-zz]

Following was the response of @CarnaViire
From what I understand from your usage, you need to have several clients with different configurations, that you might need to inject into singletons. In that case, you can either:

1. Let HttpClientFactory cache them for you, and fix singleton usage by setting PooledConnectionLifetime and disabling rotation in the factory, as described in the issue above. In that case, you don't need to have additional cache on your side.

2. You can also opt for original usage pattern of HttpClientFactory, where you would inject not the client, but the factory itself in a singleton service and call factory.CreateClient(name) every time you need one, so the factory can handle the caching and rotation (in that case you would not need PooledConnectionLifetime). Here, you also don't need an additional cache.

3. If you don't actually need configurability HttpClientFactory offers, you can do the other way around: you can have only your singleton cache, where you would create HttpClients (with PooledConnectionLifetime) by yourself and then hold them throughout lifetime of your application.

[akshay-zz]

services.AddHttpClient(name, options =>
        {
            options.BaseAddress = new Uri(baseUrl);
        })
        .ConfigurePrimaryHttpMessageHandler(() =>
        {
            return new SocketsHttpHandler()
            {
                PooledConnectionLifetime = TimeSpan.FromMinutes(pooledConnedtionLifetimeInSeconds),
                PooledConnectionIdleTimeout = TimeSpan.FromMinutes(pooledConnectionIdleTimeout),
             
            };
        });

If I'm injecting IHttpClientFactory to my scoped/transient services and creating a client by factory.CreateClient(name). HttpFactory will reuse the HttpMessageHandler internally and will respect the pool time that has been configured for the named client.

Also disposing of the Httpclient will not harm in terms of resources as we have configured in the pool time. Am I correct?

@CarnaViire

[CarnaViire]

HttpFactory will reuse the HttpMessageHandler internally and will respect the pool time that has been configured for the named client.

Yes.

disposing of the Httpclient will not harm in terms of resources

Yes, but not because of the configuration. HttpClients created by HttpClientFactory don't own the handler, HttpClientFactory does. Because of that, disposing these clients will not trigger disposing of the handler. Factory will handle the disposal when handler's lifetime expires and there's no HttpClients using it anymore.

Note that setting up PooledConnectionLifetime makes HttpClientFactory's handler lifetime (2 mins by default) redundant in this scenario (as described in #74173), so I was suggesting to turn it off (this is p.1 in my answer above)

services.AddHttpClient(name, options =>
    {
        options.BaseAddress = new Uri(baseUrl);
    })
    .ConfigurePrimaryHttpMessageHandler(() =>
    {
        return new SocketsHttpHandler()
        {
            PooledConnectionLifetime = TimeSpan.FromMinutes(pooledConnectionLifetimeInMinutes)
        };
    })
    .SetHandlerLifetime(Timeout.InfiniteTimeSpan); // Disable rotation, as it is handled by PooledConnectionLifetime

P.S.:

injecting IHttpClientFactory ... and creating a client by factory.CreateClient(name).

As long as you don't capture HttpClients in a singleton (meaning, you always create them by factory.CreateClient(name)), that would work properly even without PooledConnectionLifetime (this is p.2 in my answer above)

services.AddHttpClient(name, options =>
    {
        options.BaseAddress = new Uri(baseUrl);
    })
    .SetHandlerLifetime(TimeSpan.FromMinutes(pooledConnectionLifetimeInMinutes));

[davidfowl]

@CarnaViire is this worth documenting?

[Shayan-To]

@CarnaViire Have you deleted your answer? This thread is very confusing. I have read it a few times, and I cannot seem to find some of the references.

(This whole HttpClient and factory thing is too much confusing. I have many years of experience with .Net and front-end, and I have spent several hours across several sessions to understand how to use them, and after reading multiple blog posts which added to the confusion, and searching through the discussions here and reading this thread and others; reading this cleared things a bit for me.)

[CarnaViire]

@Shayan-To The answer was initially here #74173 (comment)

[Shayan-To]

Wow! That was very confusing!! 😅

[LeaFrock]

Have you ever read this document? For ASP.NET Core apps, it's relatively simple to reach the best practice. I prefer the Typed clients pattern.

But if you're developing a client app, especially a web crawler, you have to deal with some situations by yourself and give up some guidance rules. I think the answer from @CarnaViire is clear and concrete enough .