MQTT has some problems

[Tancen]

In RabbitMQ 3.8.2, Erlang 22.2.7

This queue has 300+ messages not consume, And has 0 consumers.
The first message is :

In actuality, One consumer exists, The first message has already been consumed (All the messages have been consumed ).

[michaelklishin]

I will convert this issue to a GitHub discussion. Currently GitHub will automatically close and lock the issue even though your question will be transferred and responded to elsewhere. This is to let you know that we do not intend to ignore this but this is how the current GitHub conversion mechanism makes it seem for the users :(

[michaelklishin]

I don't understand what the problem is as not having consumers is not a behavior of RabbitMQ and can be a natural state of your system.

3.8.2 is over two years and 20+ patch releases old.

[Tancen]

I don't understand what the problem is as not having consumers is not a behavior of RabbitMQ and can be a natural state of your system.

3.8.2 is over two years and 20+ patch releases old.

I'm sure that all messages are consumed, But the console indicates that there are many messages left unconsumed, And in fact, at least one consumer exists.
Thank you for your suggest, We will try to update the program to verify that the problem is resolved.

[Tancen]

I tried to update the RabbitMQ version to 3.9.13, This problem still exists. The count of consumers isn't possible 0, Because it's MQTT.

[Tancen]

I don't understand what the problem is as not having consumers is not a behavior of RabbitMQ and can be a natural state of your system.

3.8.2 is over two years and 20+ patch releases old.

Sorry, I was wrong, This MQTT queue is persistent.

[lukebakken]

Please provide a way for us to reproduce this issue.

[Tancen]

Please provide a way for us to reproduce this issue.

I looked at some documents, MQTT says that if the 'cleansession' is false, and the client ID is unique, the session is persistent. Otherwise, the session is clean.
In the previous picture, The 'auto-delete' is 'true'(clean session) and the count of consumers is '0'. It's not possible for MQTT.
I tried to reproduce this problem, But it's so hard, It does not occur every time, I don't know what conditions will reproduce this problem.

[lukebakken]

I'm sure that all messages are consumed, But the console indicates that there are many messages left unconsumed, And in fact, at least one consumer exists.

Without a way for us to reproduce this issue, we can't assist you. We need detailed instructions or, ideally, a script that can be run to demonstrate what you are seeing.

[Tancen]

I'm sure that all messages are consumed, But the console indicates that there are many messages left unconsumed, And in fact, at least one consumer exists.

Without a way for us to reproduce this issue, we can't assist you. We need detailed instructions or, ideally, a script that can be run to demonstrate what you are seeing.

You can check the implementation about QoS, I discover that flag 'D' with if QOS value is '1', flag 'AD' with if 'cleansession' value is true. If the QoS is 1 and 'cleansession' is true, The queue with flags 'D AD'. All bad queues are with flags 'D AD'. All good queues are only with flag 'AD'.

[Webbeh]

Hey there.

We have several "similar" issues with MQTT queues. The publisher devices do not support CleanSessions (ADFWeb MBus <-> MQTT), which I suppose is the issue we're facing here.

We are publishing from those devices with a QOS of 0, and subbing with a QOS of 0 from a node-red instance.
Some queues use wildcard subscriptions, some don't.

All queues having those issues have the "D AD" flags, and get very big, very fast.

Oddly enough, even with both pub and sub being QOS 0, the queue is named QOS 1 automatically.

The 2nd queue in that screenshot has the same settings as the other ones. I don't understand why it doesn't have the Auto-delete flag as well...

Do you think we would have less issues with clean sessions ?

I will admit we're on a quite old version of RabbitMQ (3.7.8-4) and it could very well be what's causing our issue, but we sadly have a very slow IT department. I apologize in advance if my issues are fixed in another version.

[Webbeh]

I see however that the "subscription_ttl" setting applies to the queue. We always have active consumers (node-red is always on listening to every message arriving), so the queue itself needs not expire. However, the messages need to.

I will apply it to the messages TTL.

Thank you !

[Webbeh]

Alright. So the problem goes slightly deeper.

• The queue TTL works as intended. If a queue is inactive, it is deleted
• The message TTL has absolutely no effect. The queues keep getting bigger.
• With Node-Red, a new queue is created per "connection" used, and every topic subscribed on that "connection" is part of that queue. If ONE of those topics is subbed as QOS1 (or QOS2), the whole queue becomes QOS2
• When you change a sub from QOS2/1 to QOS0, a new queue is created correctly, but the old routing key is still valid in the previous queue and needs to be manually unbound

The last point is what was happening to our server. Several previously QOS1 items were still bound to that queue, hence it kept growing. We unbound everything and restarted rabbitMQ, everything seems to be back to normal (besides a few unclean session devices)

Thanks for your help !

[michaelklishin]

The docs explain how message TTL works and when those expired messages are actually acknowledged and marked for deletion. It's when they reach the head of the queue that has a consumer. They won't be delivered to that consumer but queues that do not "move" do not delete anything either.

[Webbeh]

I guess it makes sense. However, what does not make sense is that the routing keys are not removed from a queue when a MQTT connection is disconnected.

I understand why it happens, but I don't understand any possible use that this could have.

[michaelklishin]

A routing key is not "attached" to a queue, it's a property of either a message published or a binding (routes).

Bindings can be removed in case of a clean client disconnections but it's extremely common for MQTT clients to just close TCP connection. We cannot know if that was intentional or not, and deleting bindings that back MQTT subscriptions of that client is therefore not safe.