Skip to content

Quorum queue delivery count set to unlimited can cause the number segment files to increase without truncation #14202

@Mott88

Description

@Mott88

Describe the bug

As described in the documentation you can set the delivery limit for quorum queues to unlimited for rare use cases: https://www.rabbitmq.com/docs/quorum-queues#disabling-the-limit
"The old unlimited delivery-limit behaviour can be restored by setting a queue argument or policy with a delivery limit of -1. It is not recommended to do so but may be needed for 3.13.x compatibility in some rare cases."

If you add a policy that sets the delivery limit for a quorum queue to -1, RabbitMQ starts re-applying policies to the queue every 5 second, which causes the number of segment files to increase because there is added a "update_config" log event every time re-apply policies is run.

I'm no developer and have little Erlang experience, but I think this happens because the maybe_apply_policies is run every tick

maybe_apply_policies(Q, Overview),

Then it seems like the -1 in deliverylimit causes the NewPolicyConfig not to be equal to CurrentPolicyConfig" which triggers the update_config event:

ShouldUpdate = NewPolicyConfig =/= CurrentPolicyConfig,

Reproduction steps

  1. Install RabbitMQ 4.1.2
  2. Add raft.wal_max_size_bytes = 1000 to rabbitmq.conf to make the write-ahed-log rotate quickly.
  3. Start RabbitMQ.
  4. Create a single quorum queue named q1 and publish a message to the queue.
  5. Enable debug logging for RabbitMQ.
  6. Add a quorum queue policy that sets the delivery limit for the queue you created to -1.
  7. Log entries with " Re-applying policies to queue" will start to show up in the log every 5 second.
  8. The write-ahed-log will start to grow every 5 second, because there is added an update_config log event everytime it is trying to re apply policies. When the wal gets rotated these events is added to the queue segment files. If the queue has no traffic (no active consumers or messages published), the number of segment files will continue to increase with 4096 update_config events for each segment file. To list the number of update_config events in a segment file run the command strings 0000000000000002.segment | grep -o "update_config" | wc -l

Expected behavior

Don't expect RabbitMQ to apply the policy every 5 second.

Additional context

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions