-
Notifications
You must be signed in to change notification settings - Fork 405
Open
Description
Hi there, I believe this is a bug in the node-rdkafka library. When we use cooperative rebalancing, on shutdown some consumers return the rebalance protocol as NONE rather than COOPERATIVE.
Environment Information
- OS [e.g. Mac, Arch, Windows 10]:
- Node Version [e.g. 8.2.1]: 22.22.0
- NPM Version [e.g. 5.4.2]: 10.9.4
- C++ Toolchain [e.g. Visual Studio, llvm, g++]:
- node-rdkafka version [e.g. 2.3.3]: 3.6.0
Steps to Reproduce
- Create a KafkaConsumer with partition.assignment.strategy: 'cooperative-sticky' and rebalance_cb: true
- Connect the consumer and wait for it to join a consumer group and receive partition assignments
- Call consumer.disconnect() (or stream.destroy() on a KafkaConsumerStream)
- Observe that the final revoke rebalance callback fires, but rebalanceProtocol() returns 'NONE' instead of 'COOPERATIVE'
- It looks like the built-in handler calls unassign() instead of incrementalUnassign(), which throws inside librdkafka. The error is silently swallowed because isConnected() is also false at this point. disconnect() never completes.
node-rdkafka Configuration Settings
'partition.assignment.strategy': 'cooperative-sticky',
rebalance_cb: true,
Additional context
We observe this happening intermittently, when we have multiple consumers with a large number of partitions. When rebalancing happens, only some consumers return 'NONE' instead of 'COOPERATIVE'.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels