[Questions] How to measure whether a queue is in a high or low pressure state

[tommwwu]

Community Support Policy

• I have read RabbitMQ's Community Support Policy
• I run RabbitMQ 4.x, the only series currently covered by community support
• I promise to provide all relevant information (versions, logs from all nodes, rabbitmq-diagnostics output, detailed reproduction steps)

RabbitMQ version used

4.0.5

Erlang version used

27.2.x

Operating system (distribution) used

linux

How is RabbitMQ deployed?

Community Docker image

rabbitmq-diagnostics status output

None

Logs from node 1 (with sensitive values edited out)

None

Logs from node 2 (if applicable, with sensitive values edited out)

No response

Logs from node 3 (if applicable, with sensitive values edited out)

No response

rabbitmq.conf

I'm not sure about the cluster configuration

Steps to deploy RabbitMQ cluster

My question doesn't require me to provide this information

Steps to reproduce the behavior in question

None

advanced.config

No response

Application code

No response

Kubernetes deployment file

No response

What problem are you trying to solve?

How does rabbitmq calculate the consumption rate of a queue, and what metrics should I base my calculations on if I want to measure how stressed a queue in my cluster is?

[michaelklishin]

By incrementing certain counters when a basic.deliver and similar frames (depending on the protocols) are sent out.

I don't know how you define "queue stress". The metrics offered by RabbitMQ's Prometheus scraping endpoint and HTTP API are pretty standard for a messaging and streaming system.
Ingress and egress rates, number of messages ready for delivery, number of messages, consumer acknowledgement rates are all pretty standard per-queue metrics to make decisions on.

With a large number of queues you cannot practically rely on per-queue metrics, only on aggregated ones.

There's also a consumer utilization which was renamed but is still as relevant for detecting slow consumers, an insufficient number of them or a low prefetch value used by consumers.

There's no shortage of other metrics, including from the runtime, infrastructure and so on.

Finally, if you don't know what a node, specific queue or the cluster with N clients can demonstrate as a baseline, you cannot really reason about the level of "stress". Use PerfTest, Stream PerfTests for that.

[tommwwu]

All I need at the moment is to look at the load of one queue at a time, not the whole cluster or a particular node.
As for how to judge this pressure, I simply think that the queue length is long and the consumption rate is slow, must be less pressure than the queue length is short and the consumption rate is fast, but this judgement I personally think it is too one-sided!

[michaelklishin]

Then see what the available metrics are (in Prometheus but also from GET /api/queues/{vhost}/{name}).

DO NOT use GET /api/queues to fetch one metric of one queue, there are 70 or so metrics per queue rendered and it would be extremely wasteful.

[tommwwu]

thanks!