Quorum queue declaration returns before a leader is elected

[illotum]

An interesting race condition came to my attention.

New queue declaration doesn't wait for Ra to finish election, so clients fail unexpectedly when publish to the queue immediately after. This is within Raft semantics, but a surprising behaviour to outsiders.

HTTP API and CLI all have client-side timeouts allowing us to make them wait for the election, but I wonder if anyone has a suggestion on how this can be done for AMQP?

Example code this leads to

private void declareDurableQueue(final Channel channel) throws IOException {
	final Map<String, Object> args = ImmutableMap.of(RABBITMQ_QUEUE_TYPE_ARG, queueType.getValue());
	channel.queueDeclare(queueName, true, false, false, args);
	if (queueType == RabbitMqQueueType.QUORUM) {
		log.info("Wait {}s for QUORUM queue {} to elect a leader.", QUORUM_QUEUE_STABILIZATION_DURATION.getSeconds(), queueName);
		Thread.sleep(QUORUM_QUEUE_STABILIZATION_DURATION.toMillis());
	}
}

[michaelklishin]

@illotum the only option is to make the queue.declare operation implementation await leader election for QQs. It can, as you know, take seconds or tens of seconds in some cases, so I don't know which behavior sounds better to me.

I guess a potential timeout makes more sense.

[illotum]

Catching up on AMQP: can we rely on nowait as escape hatch? If the client app can decide whether to block or not, this seems to be fairly straightforward.