Skip to content

Commit d72a192

Browse files
committed
Avoid 100% CPU usage while socket is closed
After stop/start kafka service, kafka-python may use 100% CPU caused by busy-retry while the socket was closed. This fix the issue by adding a 0.1 second sleep if the fd is negative.
1 parent 316da74 commit d72a192

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

kafka/client_async.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -634,6 +634,16 @@ def _poll(self, timeout):
634634
self._sensors.select_time.record((end_select - start_select) * 1000000000)
635635

636636
for key, events in ready:
637+
if key.fileobj.fileno() < 0:
638+
# if there is no new message, self._selector.select() will take 0.1s.
639+
# However, if the socket is closed, it could return immediately, result in
640+
# high CPU usage because of busy-retry, caused by the loop in
641+
# producer/sender.py run()
642+
# calling producer/sender.py run_once()
643+
# calling lient_async.py poll()
644+
# calling lient_async.py _poll()
645+
time.sleep(0.1)
646+
637647
if key.fileobj is self._wake_r:
638648
self._clear_wake_fd()
639649
continue

0 commit comments

Comments
 (0)