Don't wait for RPC response. Implementing RPC asynchronously?

[WilliamDEdwards]

I have the following situation:

• A message will trigger a long-running job on the consumer
• The consumer should publish the result
• The publisher of the original message should be able to check if there was a result at any time, without the call being blocking

With the 'normal' RPC pattern, waiting for the result is blocking:

while self.response is None:
    self.pika.connection.process_data_events()

return self.response  # type: ignore[unreachable]

The RabbitMQ documentation says:

If you can, you should use an asynchronous pipeline - instead of RPC-like blocking, results are asynchronously pushed to a next computation stage.

Instead of consuming messages on the callback queue right after publishing (blocking), I could return the callback_queue and correlation_id immediately, and save it somewhere. When requesting the result, I would:

• Retrieve the callback_queue and correlation_id
• Consume messages on the callback queue
• Wait until a message arrives with the correlation_id. To ensure that we don't wait for the results indefinitely, I'd set call_later on the Pika connection, like:

TIMEOUT = 2

pika.connection.call_later(TIMEOUT, timeout)

def timeout() -> None:
    """Raise exception when no RPC response has been received."""
    raise Exception(f"Did not receive response in {TIMEOUT} seconds, try again later")

So basically, I publish the message, but instead of waiting for the result, save the callback_queue and correlation_id so that the client may retrieve the results at any time.

Note: the callback queue is currently declared as exclusive, meaning only the original publisher connection may consume from it. In this case, separate Pika connections would consume from it, so I'd have to delete it myself after getting a result.

Is this a generally accepted way to implement RPC asynchronously?