feat: support blocking send_non_blocking() and other RPCs#355
feat: support blocking send_non_blocking() and other RPCs#355BewareMyPower merged 6 commits intomasterfrom
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for blocking behavior when the internal pending queue is full during send operations, addressing issue #342. The change allows users to choose between blocking and non-blocking behavior when the outbound channel reaches capacity.
- Adds
block_queue_if_fulloption toProducerOptionsto control blocking behavior for send operations - Modifies connection logic to differentiate between send RPCs (which can fail fast) and other RPCs (which should block)
- Enhances documentation for the new option and existing
send_non_blockingmethod with usage examples
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/producer.rs | Adds block_queue_if_full option, updates documentation with usage examples, and includes comprehensive tests |
| src/connection.rs | Refactors message sending logic to support both blocking and non-blocking modes based on the new parameter |
| src/client.rs | Adds documentation for the with_outbound_channel_size method |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
src/producer.rs:149
- Typo in comment: 'treshold' should be 'threshold'.
/// batch size in bytes treshold (only relevant when batch_size active).
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
Thank that a great addition! |
Fixes #342
Motivation
#312 changes connection's outbound channel from unbounded to bounded and once the channel is full, all RPC requests will fail with the
SlowDownerror. However, other RPCs likeAckcould also be affected. This is different from other Pulsar client SDKs because only pendingSendRPCs should be limited.This behavior is not documented and hence very confusing.
Modifications
Add a
block_if_queue_fulloption to simulate the feature from Java client:https://github.com/apache/pulsar/blob/14543d3fde935d1b70e3707a8b2c0294eb53dccb/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerBuilder.java#L241
SendRPCasync_channel::Sender::sendwhen this option is enabled (disabled by default)async_channel::Sender::try_sendand fail withSlowDownif queue is fullasync_channel::Sender::sendAdd
block_if_queue_fullto show the difference between the option is enabled and disabled.Additionally, add documents to this new option, as well as
send_non_blockingbecause it's hard for users to know how to handle thisSlowDownerror without the documentation.