Consumer .poll() method blocks everything.

[aascedu]

Hello,

I think I am not facing a package issue but rather a skill issue.

I don't understand why .poll() is blocking my program. When I run the code below, nothing happens on the output however when I add the sys.stdout.flush() calls, it starts printing.
Here is the very basic program :

from confluent_kafka import Consumer, KafkaError
from confluent_kafka.admin import AdminClient, NewTopic
import sys

bootstrap_servers = 'broker-1:19092,broker-2:19092,broker-3:19092'

conf = {'bootstrap.servers': bootstrap_servers,
        'group.id': 'availability-tool',
        'enable.auto.commit': 'true',
        'auto.offset.reset': 'earliest'}
consumer = Consumer(conf)
TOPICS = ['asset_state']
consumer.subscribe(TOPICS)


admin = AdminClient({'bootstrap.servers': bootstrap_servers})
print("List of topics in Kafka")
for topic in admin.list_topics().topics:
    print(f"{topic}")

print(f"List of topics for {consumer}")
for topic in consumer.list_topics().topics:
    print(f"{topic}")

# sys.stdout.flush()
while True:
    msg = consumer.poll(2.0)

    if msg is None:
        print("...")
        continue
    if msg.error():
        print("Consumer error: {}".format(msg.error()))
        continue

    print('Received message: {}'.format(msg.value().decode('utf-8')))
    # sys.stdout.flush()
consumer.close()

I tested my program with kafka official console binaries to see that messages where sent to the consumer and they were.
What does the poll() method do that I don't understand ?

I did restart multiple times my program but even after a clean start, it isn't working

Thanks in advance.

[aascedu]

Also, the most optimal way I found to have efficient output is this :

while True:
    sys.stdout.flush() # call to stdout flush() HERE
    msg = consumer.poll(2.0)

    if msg is None:
        print("...")
        continue
    if msg.error():
        print("Consumer error: {}".format(msg.error()))
        continue

    print('Received message: {}'.format(msg.value().decode('utf-8')))
consumer.close()

If anyone have some time to help me figure it out that would be nice !

[aascedu]

print call is stuck, using a logger fixes it. So poll() is not at fault.