Limits of routing keys per queue

[pmajcherssh]

Community Support Policy

• I have read RabbitMQ's Community Support Policy

RabbitMQ version used

4.0.3

How is RabbitMQ deployed?

Community Docker image

Steps to reproduce the behavior in question

Hello Rabbit Community,

I'm designing a backend for an IoT system, and I'd like to set up a dedicated routing key for every instance of the IoT device. Are there any limits for the number of bindings inside a single queue? Are there any good practices in that matter?

I'd like to have hundreds of thousands of different routing keys per single queue. I will perform performance tests of this setup, but also wanted to make sure that what I'm doing is not against the way the routing key were designed.

I wonder how does Rabbit look for the proper routing key. Is there some internal "DB" with indexes inside to make it quick?

Best regards,
Piotr Majcher

[mkuratczyk]

What protocol are you going to use? This affects your options.

Hundreds of thousands are generally possible (we are aware of some installations with such numbers), although not exactly popular. ;)

Some options you might consider:

1. Using the default direct exchange (with an empty name) - it's a special exchange which doesn't require any bindings - it just assumes the routing key is equal to the queue name.
2. Using a fanout exchange - you said a lot of bindings per queue, so perhaps all you need is just a fanout exchange, which ignores the routing key and will route the messages to the same queue anyway?

[pmajcherssh]

Hello @mkuratczyk thanks for a prompt reply.

I wanted to use direct exchange with bindings.

Please take a look at this diagram

As you can see on the diagram, I need to be sure, that no matter which backend on the left side handles the request event from, for instance iot_device_1, the response event has to be routed to the WS to Rabbit Adapter that holds the WebSocket connection to the iot_device_1, in this case WS to Rabbit Adapter 1. That's why I thought, let's create a binding (using device id as a routing key) every time a new device is connected to the Adapter. That will allow me to create a dedicates pipe for this specific device.
Of course the diagram is a bit simplified, all backend apps should point to a single queue and just specify a routing key, not as it's shown on my diagram, routing should be done on the Rabbit side.

There is a load balance between the Adapters layer and the devices, so I never know which Adapter holds the WS connection.

Getting back to your suggestions, I can't use fanout, because I want the message to be delivered to a concrete consumer, not to all.

[michaelklishin]

Use a topic exchange (with MQTT, AMQP 0-9-1, AMQP 1.0, STOMP) and/or permissions for individual queues. Queue permissions are less practical if the number of devices is expected to be, say, in tens of thousands, then topic permissions and a smaller number of queues would make more sense.

[michaelklishin]

There are no routing key permissions but there are topic permissions as well as permissions to individual queues.