add support for QUIC protocol

[trevorboydsmith]

Is your feature request related to a problem? Please describe.

Is your feature request related to a problem?

yes:

• when you want/need better performance (throughput and/or latency)
• when you need to scale up
• when you need to have multiple AMQP channels on a TCP connection that uses an unreliable/slow connection (for example cellular connection with bad service and/or moving cellphone)
• better/easier TLS because it is taken care of by QUIC

high level reasons/motivation:

the same reasons/motivation that Google has developed/implemented/deployed QUIC protocol all apply to Rabbitmq... i.e. better performance (throughput, latency), more reliability (one lost QUIC packet does not freeze all the Rabbitmq Channels), probably uses less server side resources (cpu). for more on the motivation see: https://datatracker.ietf.org/doc/rfc9000/ ; also read about why Google developed and deployed and helped get the protocol specification written and adopted for use in HTTP version 3; also see https://quicwg.org/ .

more specific to RabbitMQ:

QUIC's implementation of multiple parallel data streams seems directly analogus to RabbitMQ's multiple Channels over one connection... so QUIC seems like an obvious and significant step forward for RabbitMQ.

Describe the solution you'd like

the same RabbitMQ server but replace all TCP packets with QUIC packets (and let QUIC do the TLS instead of having RMQ need to do a custom implementation).

do some performance benchmarking afterwards to see how much better performance or how much less CPU resources are needed.

Describe alternatives you've considered

When people have trouble with RabbitMQ performance or scaling, I have heard people abandon RabbitMQ for a different product like: ZMQ, Kafka, Pulsar, Redis.

But IMO having RabbitMQ use QUIC may significantly increase RabbitMQ performance and then you could continue using RabbitMQ.

Additional context

everyone already uses a client library like Pika for Python or librabbitmq for C/C++ and so switching the server underlying protocol from TCP to QUIC could be straightforward... like a boolean in the rabbitmq.conf that says to use QUIC and then all clients have to use a QUIC version of the client code... but all the Pika API stays the same.

[michaelklishin]

Google were solving a completely different problem and their starting point was HTTP 1.

When you need better throughput, RabbitMQ streams provide millions of messages per stream with just three replicas (up to a certain point, of course). I am not going to get into benchmarketing but
we know first hand that RabbitMQ streams, when multiple are used with their respective Stream protocol clients, beats some of the solutions mentioned above.
Without using QUIC.

Sorry but I cannot take claims like "everyone does X", "Y should be easy", "do N and the performance will be amazing, I promise (but you do all the work)" seriously.

In particular this claim

everyone already uses a client library like Pika for Python or librabbitmq for C/C++ and so switching the server underlying protocol from TCP to QUIC could be straightforward

is absolutely not true because all four protocol key RabbitMQ supports (AMQP 0-9-1, AMQP 1.0, MQTT, STOMP, RabbitMQ Stream Protocol), all are TCP-based and so are their clients. There are dozens of protocols that would have to be updated to use both TCP and QUIC (QUIC alone will not fly).

[trevorboydsmith]

thanks for the answer! my intent was to share an idea for improving RabbitMQ. i do recognize that there would be significant amounts of work on both the server side and client side (especially given that RMQ supports not just AMQP 0-9-1, AMQP 1.0, STOMP, RabbitMQ Stream).

thanks and have a great day.

[michaelklishin]

One area where QUIC support would be relatively effortless to achieve, and would make most sense if you ask me, is support for HTTP/3 for publishing of messages.

Publishing over the HTTP API was never meant to be used in production but it's not unheard of to see this.

Cowboy, the HTTP server RabbitMQ uses, has HTTP/3 work under way ninenines/cowboy#1544.

Given a suitable client, this should be another reasonably efficient publishing option. Or at least that one that looks promising.

I doubt it will beat the Stream
Protocol because key gains there come from parallelism (clients connect to N nodes at the same time) and batching.