Deliver a message to a different consumer on basic.reject
#9913
-
Is your feature request related to a problem? Please describe.As detailed in this blog post from August 2010 and the AMQP specification overview RabbitMQ has decided not to implement the following part of the spec:
The blog post suggests that this is useless when this statement is true at the same time:
However, I think this former rule is still useful for messages that cannot be processed at the current moment due to a lack of enough resources. This means the client is not selecting messages based on the content of messages alone, but on the content of the message in combination with the currently available resources which change over time. I think a common use case for rejecting messages could be to check for available capacity of some resource upon receiving a message and Additionally, we currently observe that RabbitMQ does not only not prevent the message being delivered to the same client, but quite the opposite: The message is delivered to the same client almost every time making the load balancing very inefficient. Describe the solution you'd likeImplement a best effort solution to select a different consumer for the message if one is available to partly implement the spec. If no consumer is available the message could be delivered to the same consumer like in the current solution. In the best case, users would be able to specify a time period that RabbitMQ waits for another consumer to become available before it is delivered to the same consumer as it is suggested in the spec:
Describe alternatives you've consideredPublishing the message to the back of the queue, which is not desirable if there are other consumers that could handle the message at the current moment. Additional contextNo response |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 1 reply
-
|
@mlichtblau this is not worth the effort. The complexity of selecting an "available" consumer (how do we define that?), the subtle change in behavior, and the fact that RabbitMQ's small core team have dozens of more important things to worry about, all make this a mostly impossible ask. |
Beta Was this translation helpful? Give feedback.
-
|
In fact, RabbitMQ has dropped at least one feature related to consumer locality that the AMQP 0-9-1 spec has: the Making RabbitMQ depend on consumer state in general sounds like a terrible idea, given our yeas of experience. Applications are developed without metrics and with very poor error handling all the time, and RabbitMQ usually gets the blame for that. |
Beta Was this translation helpful? Give feedback.
-
|
Implementing this is indeed very complex and not something we get a lot of requests for. There is a good argument to try to keep queue implementations (we now have 2 or 3 independent implementations depending on how you count) as simple as possible. As you observed if at the time the of the reject there are no additional consumers with capacity (unmet prefetch) it will be re-delivered back to the original consumer. The only alternative I can think of is that if the consuming app detects it has insufficient resources available to process it can unsubscribe or shut down it's channel and enter a polling loop to detect when the resource is available again and resubscribe at that point. |
Beta Was this translation helpful? Give feedback.
-
^^^^ 💯 👍 |
Beta Was this translation helpful? Give feedback.
Implementing this is indeed very complex and not something we get a lot of requests for. There is a good argument to try to keep queue implementations (we now have 2 or 3 independent implementations depending on how you count) as simple as possible.
As you observed if at the time the of the reject there are no additional consumers with capacity (unmet prefetch) it will be re-delivered back to the original consumer.
The only alternative I can think of is that if the consuming app detects it has insufficient resources available to process it can unsubscribe or shut down it's channel and enter a polling loop to detect when the resource is available again and resubscribe at that point.