MQTT client publishing to MQTT topic bound to a super stream

[ricardoduncan]

Community Support Policy

• I have read RabbitMQ's Community Support Policy

RabbitMQ version used

4.0.2

How is RabbitMQ deployed?

Kubernetes Operator(s) from Team RabbitMQ

Steps to reproduce the behavior in question

The native MQTT plugin for RabbitMQ has been enabled in a Rabbit cluster.

The design goal is to enable MQTT clients to publish, with QoS 0, to an MQTT topic that is bound to a super stream within Rabbit. The goal is to partition the super stream across X number of streams to scale consumers subscribing to the streams using the stream protocol.

RabbitMQ recommends, as best practice, that stream producers connect to the stream leader node in a cluster setup to minimize latency. Clients using the stream protocol can use the metadata command to discover stream topology (which node is the leader and which ones are followers).
However, in this case the client is an MQTT publisher and hence not using the stream protocol. How can this MQTT client ensure it connects to the stream leader node in the Rabbit cluster?

[ansd]

Publishing locally to the queue/stream leader is supported in the Stream protocol and in the AMQP 1.0 Java client.

For MQTT such a feature could supposedly be implemented via https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901255 and the client requesting affinity to a specific queue/stream leader/replica in https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901054

However, we currently do not plan to implement this feature for MQTT.

[ricardoduncan]

Thanks for your response. We will probably need to POC how much network load can arise when publishing MQTT clients are not connected to the stream leader node in our setup. In our case all nodes are in the same data center, so I expect inter-node latency to be sub 5ms