Ordered Subscriber receiving only one call back per ack

[pantoss]

Can anyone help me out. I cannot see where I'm going wrong. I've got a very simple process which creates a subscriber to a topic where the messages are published with an ordering key. I want to receive a batch of the messages, process them and then call ack for the batch all in one go. Processing the messages in a batch will be more efficient but I'm finding that the receive call back is only called once and isn't called again until an ack is sent.

More details of the code. The receive callback places the message and ackhandler onto a thread safe queue to be process by a separate thread. The separate thread is waiting on that queue and will take what ever messages our on that queue, process them and then call ack for all of them. When the messages have been placed on the topic with a single ordering key there will only ever be a single message received prior to the ack being call. If multiple ordering keys are used it will have multiple messages returned prior to ack being called.

I've tried to set the
set_max_outstanding_messages(1000).set_max_outstanding_bytes(16 * 1024)
subscriber options but it doesn't seem to make a difference.

I cannot find anything which suggests to me that this is intended behaviour but also I cannot see what I might be doing wrong either.

I do seem to be using an old version of the api 1.24. incase it is old behaviour. If this should work can anyone point me to some example code which performs what I expect so I can see the difference with my own code please?

[pantoss]

I think I've found the answer to my own question. Incase it is helpful for others in future.

There is a single runnable_messages_ queue and multiple threads read from that single queue it has to wait until the HandleDone has happened before placing the next message on the runnable_messages_ queue as it cannot be certain what order those runnable_messages_ might end up being processed in by the multiple threads.

From my perspective it would be nice to separate the action of ack/nack from completion of the receive call back so that the receiver can offload the work to be aggregated into a batch of work. For examine have a BatchAckHandler which allowed you to ack a message within a batch but it is not acked until some later point.

Maybe there is a way and i've just not see it?

[coryan]

There is currently no API to process messages in batches, nor to ack them in batches. AFAIK, all Pub/Sub client libraries do the same thing, and the rationale includes:

• Pub/Sub does not guarantee that any batch published into the service are preserved. That is, a sender could publish a batch and that could be delivered as multiple smaller batches (or vice-versa).
• There is no additional billing for sending acks or nacks.
• The general case (multiple ordering keys in a single batch, maybe with the same keys appearing more than once) requires breaking the batch per key and then reassembling.

I understand that there is an additional CPU cost involved in making more RPCs. Maybe we should consider batching the acks, on the other hand, this may be not trivial as the leases would need to be extended while the batch is being formed (maybe, leases are much longer, maybe a batch formed in a few milliseconds is enough).

[pantoss]

Thank you for your replay.

I'm not concerned about the batches they were produced in. The idea is that some efficiencies can be gained on my side to process multiple messages at the same time but if my processing should fail for any reason prior to doing my database commit I want to receive the messages again. So I need to receive a batch of messages in order, perform my processing, apply all my database updates in one go, do a single database commit and only then acknowledge the messages. I would be perfectly happy for having to acknowledge each individual messages still. There might be efficiencies to be had with batching of the acks but that isn't my primary concern at the moment, the processing and database transactions are orders of magnitude more significant than the cost of sending the acks back.

Thank you for your time.

[coryan]

I see. That seems like an interesting use-case. I can see why it would be useful to process all messages in a batch. Let me ask around.

Meanwhile: is it necessary for the messages to be received in order? Most databases can reorder things, you could create a subscription without ordering (for the same topic) and process the messages in batches using that subscription.

[pantoss]

They are not commutative operations in the messages unfortunately. For each batch we will read in some state from the database and then apply the changes in those messages to that state in memory, then write that new state to the database. It's the writing and committing that state to the database which takes the time. The messages might create an entry and then delete it for example, if they are incorrectly ordered the delete wouldn't be able to delete what it is to delete and then the create would be created when it shouldn't end up having been deleted. Overly simplified but that is the general idea.

I have asked for the publisher to use a number of different ordering keys for messages which don't have an ordering constraints but I don't have control over this. Even if they do change this, there is still efficiency to be had in the event that there are a large number of messages which are ordered received.