Question about limitations of YARP with websocket

[JuanLND]

I have a blazor server. Blazor Server by default opens a web socket through SignalR for each user, for its functionality to operate.

Without a load balancer I have a theoretical limit of 65k concurrent users as the websocket connects to each server port. But having a server that serves as a load balancer, I can increase the theoretical limit by splitting the connections between servers through the server that balances the load with YARP.

My question is, does Yarp directly bridge between my client and server in websocket calls, or is it possible to do so? Because if don't, I understand that everyone who connects to the load balancer will consume a port from the load balancer server, which in turn will broker the connection to the backend, tying my application to the server's port limit that does load balancing.

So if I have 3 servers, instead of having 65k x3 connections, I will only have 65k.

[Tratcher]

Without a load balancer I have a theoretical limit of 65k concurrent users as the websocket connects to each server port.

This is backwards. TCP connections are identified by Client IP + Port + Server IP + Port. A client uses unique client ports for each connection to the server, but the server uses the same port for all incoming connections. As such, a client is limited to 65k outbound connections to the same server IP + Port, but a server is not limited to 65k inbound connections from different clients.

Adding a proxy/load balancer actually makes the situation worse because the proxy becomes the client to the server and can only open 65k outbound connections to that server.

[JuanLND]

And how I can add the load balance/proxy YARP, to use for Blazor Applications, without compromising my situation, in other words, without having a limitation of 65k ports?

[Tratcher]

Load balancing implies you'll have multiple application instances behind the proxy? Then you'll be able to do 65k per instance. I think you can also do tricks with having multiple IPs on the proxy and app instances and cycling through them. The ConnectCallback might be the best place for that.
https://docs.microsoft.com/en-us/dotnet/api/system.net.http.socketshttphandler.connectcallback

[samsp-msft]

IIRC our websocket support is using http/1.1 and will consume the connection for the duration of the websocket lifetime, so there is not much benefit in terms of connection count by inserting a reverse proxy between a client and server.

We have an issue to implement web sockets over http/2 which would change the dynamic. The connection between the client and proxy could be used for both web socket traffic and simultaneous other http requests. Between the proxy and destination server, each http/2 connection could be used to multi-plex multiple websocket requests - potentially reducing the number of incoming connections. Most servers have limits on how many http/2 requests can be simultaneously made over one connection - this will likely need to be adjusted for websocket as it commonly defaults to 100. This may be high or low depending on how active each web socket is.

[JuanLND]

And do you know if it's possible, use yarp reverse proxy only for http, and for 'ws' do a redirect? So when the client tries to communicate with the websocket it will have a direct communication with the server, not a connection with yarp and the yarp with the server.

[Tratcher]

@JuanLND yes, you could do that in middleware.

'http' and 'ws' aren't fields actually sent in the request, a WebSocket request is an HTTP request with specific headers you could check for.

It would be something like this:

                endpoints.MapReverseProxy(proxyPipeline =>
                {
                    proxyPipeline.Use((context, next) =>
                    {
                        // https://datatracker.ietf.org/doc/html/rfc6455#section-1.2
                        // Detect WebSockets
                        if (string.Equals(context.Request.Headers[HeaderNames.SecWebSocketVersion], "13"))
                        {
                            context.Response.Redirect("https://backend" + context.Request.Path);
                            return Task.CompletedTask;
                        }

                        return next();
                    });

                    // Don't forget to include these two middleware when you make a custom proxy pipeline (if you need them).
                    proxyPipeline.UseSessionAffinity();
                    proxyPipeline.UseLoadBalancing();
                });