Cannot get rabbitmq_sharding plugin to work

[dforste]

Community Support Policy

• I have read RabbitMQ's Community Support Policy
• I run RabbitMQ 4.x, the only series currently covered by community support
• I promise to provide all relevant information (versions, logs from all nodes, rabbitmq-diagnostics output, detailed reproduction steps)

RabbitMQ version used

4.1.2

Erlang version used

27.3.x

Operating system (distribution) used

RHEL 9/Docker

How is RabbitMQ deployed?

Community Docker image

rabbitmq-diagnostics status output

See https://www.rabbitmq.com/docs/cli to learn how to use rabbitmq-diagnostics

# rabbitmq-diagnostics list_policies
Listing policies for vhost "/" ...
vhost	name	pattern	apply-to	definition	priority
/	images-shard	^shard.images$	all	{"routing-key":"1234","shards-per-node":2}	0

# rabbitmq-diagnostics list_exchanges
Listing exchanges for vhost / ...
name	type
amq.topic	topic
amq.headers	headers
images	x-modulus-hash
	direct
amq.direct	direct
amq.match	headers
amq.rabbitmq.trace	topic
amq.fanout	fanout

# rabbitmq-diagnostics list_queues
Timeout: 60.0 seconds ...
Listing queues for vhost / ...
{:badrpc, {:timeout, 60.0, "Some queue(s) are unresponsive, use list_unresponsive_queues command."}}

Logs from node 1 (with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e Server startup complete; 12 plugins started.\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039038788Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_prometheus\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039097575Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_shovel_management\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039105144Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_auth_backend_ldap\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039110842Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_federation_management\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039122934Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_federation\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039127829Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_sharding\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039133342Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_stomp\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039138327Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_random_exchange\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039143241Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_shovel\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039147338Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_management\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039160422Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_management_agent\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039164852Z"}
{"log":"2025-08-27 15:41:07.038724+00:00 [info] \u003c0.676.0\u003e  * rabbitmq_web_dispatch\u001b[0m\n","stream":"stdout","time":"2025-08-27T15:41:07.039168937Z"}
{"log":" completed with 12 plugins.\n","stream":"stdout","time":"2025-08-27T15:41:07.039172981Z"}


{"log":"2025-08-27 17:24:14.742253+00:00 [info] \u003c0.33739.0\u003e Successfully set policy 'images-shard' matching all names in virtual host '/' using pattern '^shard.images$'\u001b[0m\n","stream":"stdout","time":"2025-08-27T17:24:14.742663922Z"}

{"log":"\u001b[38;5;160m2025-08-27 17:54:10.533390+00:00 [error] \u003c0.52115.0\u003e operation queue.declare caused a channel exception not_found: no queue 'images' in vhost '/'\u001b[0m\n","stream":"stdout","time":"2025-08-27T17:54:10.533836418Z"}

Logs from node 2 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 3 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

rabbitmq.conf

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find rabbitmq.conf file location

# PASTE rabbitmq.conf HERE, BETWEEN BACKTICKS

Steps to deploy RabbitMQ cluster

services:

  rabbit1:
    image: rabbitmq:4.1.3-management
    hostname: rabbit-1
    restart: always
    ports:
      - "4369:4369"
      - "5671:5671"
      - "5672:5672"
      - "6000-6500:6000-6500"
      - "15671:15671"
      - "25672:25672"
      - "15692:15692"
    environment:
      RABBITMQ_NODE_TYPE: stats
      RABBITMQ_ERLANG_COOKIE: '....'
    volumes:
      - /app/container/rabbitmq/config:/etc/rabbitmq
      - /data/rabbitmq:/var/lib/rabbitmq
      - /etc/ccn.pem:/etc/ssl/ca.pem:ro
      - /etc/pki/tls/certs/RMQ-vfp-extlogrmq-1.ftscc.net.crt:/etc/ssl/server.pem:ro
      - /etc/pki/tls/private/RMQ-vfp-extlogrmq-1.ftscc.net.pem:/etc/ssl/server_key.pem:ro
    ulimits:
      nproc: 25000
      nofile:
        soft: 20000
        hard: 25000

Steps to reproduce the behavior in question

Added rabbitmq_sharding to the enabled_plugins file and restarted.

I tried to follow the steps on https://github.com/rabbitmq/rabbitmq-server/blob/main/deps/rabbitmq_sharding/README.md for testing.

rabbitmqctl set_policy images-shard "^shard.images$" '{"shards-per-node": 2, "routing-key": "1234"}'
Created the exchange from the management UI.

advanced.config

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find advanced.config file location

# PASTE advanced.config HERE, BETWEEN BACKTICKS

Application code

# PASTE CODE HERE, BETWEEN BACKTICKS

Kubernetes deployment file

# Relevant parts of K8S deployment that demonstrate how RabbitMQ is deployed
# PASTE YAML HERE, BETWEEN BACKTICKS

What problem are you trying to solve?

Use rabbitmq_sharding. I also don't see any queues created in the managment ui.

[michaelklishin]

The plugin works as before. We haven't touched it for a while.

All it takes is declaring an exchange of type x-modulus-hash provided by the plugin, then setting up a policy that will match it:

rabbitmqctl set_policy images-shard "^shard.images$" '{"shards-per-node": 2, "routing-key": "1234"}'

Which results in two queues per node declared by the plugin and bound to the x-modulus-hash exchange:

rabbitmq_sharding Relevance in 2025

However, in the age of streams and partitioned streams, rabbitmq_sharding is no longer necessary or relevant.

So, simply use streams or partitioned streams with a RabbitMQ Stream Protocol client. If deploying to an environment that mandates the use of proxies, such as Kubernetes, see this blog post.

That's about it.