Add support for priority queues for quorum queues before RabbitMQ 4.0

[jefftrimm]

With the removal of Classic Queue mirroring, there remains no queue types that support different message priorities. This is a critical feature for many users.

This is to request support for message priority support for quorum queues, and to do so before classic mirrored queues are deprecated in RabbitMQ 4.0 to allow for a migration process for existing priority queue users.

https://www.rabbitmq.com/priority.html

[michaelklishin]

Quorum queues were never meant to be a feature-for-feature, 100% equivalent of classic queues. The whole idea behind having queue types is that they optimize for different things. Quorum queues do not optimize for priority access, and chances are, never will. We won't repeat the same mistake of putting every conceivable feature into one queue type again.

[michaelklishin]

Non-mirrored priority queues will remain even after 4.0. It's perfectly reasonable to use a non-replicated queue type for some things, and a quorum queue or stream for other things. Supporting every possible combination of features is not a priority and likely will not be worth the engineering effort. We have ample evidence that most users do not use priority queues, and when they do, it is only to avoid messages of one type from building up behind others, which can be achieved by using a queue or group of queues per message type.

[jefftrimm]

Removing support for highly available priority queues is a pretty major deprecation of functionality that will likely impact some important existing integrations out in the wild dependent on this? Message priorities are pretty much table stakes for any modern message broker.

If it's decided to not support message priority in quorum queues, can it at least be considered to retain the ability of having mirrored queues continue to support this functionality?

[michaelklishin]

@jefftrimm classic queue mirroring is a broken and unfixable design. Willingly using it means you don't value your data that much, which means you should be able to just roll with non-mirrored priority queues which are not going to be removed.

While we cannot measure it, our team's impression of how widely used priority queues are does not match yours. Like I said, very often priority queues are used to untangle two or more message types to avoid head-of-the-queue blocking problems but not any prioritisation beyond that. This can be easily achieved by using multiple queues.

This is discussion material.

[micdah]

This can be easily achieved by using multiple queues.

I have seen this approach suggested multiple places, but as I see it there is one major disadvantage to this suggestion.

When a single consumer has to receive messages from multiple queues, the only way to do it is via polling, which adversely affects the server's performance as the high rate of poll requests generate a lot of CPU overhead.

You might suggest creating a single consumer which consumes from multiple queues - which is possible of course. But remember, we need to take from higher priority queues, before any lower - so in that case we would need to be able to instruct our RabbitMQ in which order it should deliver messages from the multiple queues the single consumer was bound to, otherwise we might be taking messages from lower priority queues while higher priority queues still have messages available.

We might have overlooked an obvious way of doing it - then I'd love to hear about it 🙈

But it seems like, the "use multiple queues" suggestion for priorities, doesn't take into account that RabbitMQ prefers (push) consumers, rather than fetching, and I see no way of using a consumer with multiple queues where we need a certain order in which messages are being delivered in relation to the queues priority in relation to other queues.

[michaelklishin]

You can reuse a consumer across queues: just call basic.consume with the same object or function. No need for polling.

[michaelklishin]

How you reorder and prioritise deliveries across those queues is up to your consumer. You get full control.