Autoscaling (with keda?)

[dj-wasabi]

How would you do autoscaling of brokers, with a tool like KEDA?
I am very curios to understand what the possibilities are. At the moment, we have a single pod (statefulset) running ActiveMQ and in the process to migrate to this operator. But I would like to understand how we can autoscale the brokers we will deploy.

We have set for some queues the address-full-policy to FAIL, but would love to see that with autoscaling we can prevent this from happening.

[gtully]

This is an open question, one potential answer is in the upstream broker test: https://github.com/apache/activemq-artemis/blob/main/tests/integration-tests/src/test/java/org/apache/activemq/artemis/tests/integration/routing/ElasticQueueTest.java#L454

at the moment, we don't have an operator that implements this strategy, but we want to get there.

With a simple cluster, and without some sharding/partitioning, messages can take multiple hops before being consumed which is less than ideal, and the queue order is compromised.
With a chain of brokers approach, order is preserved as roles are used to isolate producers and consumers.

today, federation in pull mode, would be a better approach than clustering, but it still needs some 'managed' scaling vs auto scaling to reconfigure the cooperating brokers.

[gtully]

there is a related issue that discussed scale to zero - #124 that may be of interest