RabbitMQ sharding and Consistent hash exchange: not able to distribute messages across queues uniformly

[taju20910]

Version : 3.9.7

RabbitMQ sharding plugin or consistent hash exchange is not able to distribute messages uniformly across all queues. Messages always go to one common queue among the list of all bound queues.

[michaelklishin]

I will convert this issue to a GitHub discussion. Currently GitHub will automatically close and lock the issue even though your question will be transferred and responded to elsewhere. This is to let you know that we do not intend to ignore this but this is how the current GitHub conversion mechanism makes it seem for the users :(

[michaelklishin]

We cannot suggest anything with the amount of information provided. Please help others help you and post some actual code that reproduces the behavior.

Both plugins rely on routing key values to be reasonably unique and evenly distributed or they won't be able to provide even routing target distribution. That's how hashing works: if the inputs are mostly the same, so are the outputs.

[taju20910]

Thanks Michael will look further and provide more details

[taju20910]

Please find one scenario for consistent hash with attached screenshot
I am creating 3 queues and binding them to hash_exchange with routing key 1234. when I am publishing messages in this exhange with routing key 1234 always messages go to one queue instead messages should distribute across three queues

@michaelklishin FYI

[michaelklishin]

We need executable code or scripts, not screenshots. If the routing key is identical for all messages, they all will be routed to the same queue. Again, this is how hashing works and this is documented at least for rabbitmq_sharding.

[michaelklishin]

If you want messages with the same routing key to be reasonably evenly distributed between all bound queues, see rabbitmq_random_exchange. It might need a new release for RabbitMQ 3.9 and it's hardly a perfect distribution mechanism but it does not care what the routing key is.

[taju20910]

Thanks @michaelklishin for all the help