-
Notifications
You must be signed in to change notification settings - Fork 935
Open
Description
I know, we're using an old version, but can't upgrade due to dropped CentOS support.
Really weird behaviour....
What I do is roughly:
consumer = make_consumer(enable_auto_commit=False)
consumer.subscribe(my_topic)
while True:
msgs = consumer.consume(timeout=timeout_sec, num_messages=max_records)
consumer.list_topics() # This is the crucial line - if present, next consume will be from committed offset, otherwise, will continue from last polled
consumer.commit(message=msgs[0])Somehow list_topics() ruins the expected behaviour of polling the next batch every time - in the example it polls overlapping batches.
Interesting note: if I do assign() instead of subscribe() it works as expected even with list_topics() calls.
Metadata
Metadata
Assignees
Labels
No labels