-
-
Notifications
You must be signed in to change notification settings - Fork 33.1k
Description
Crash report
What happened?
Program was running in docker container using nikolaik/python-nodejs:python3.13-nodejs23
image.
Container has been removed with it's volume as soon as process exited so I can't provide core dump.
I can't provide exact code which caused the issue because it has thousands of lines and is private.
Code heavily uses asyncio and has ~20k tasks running which are sleeping most of the time, ~20 tasks running simultaneously.
Code has many dependencies but only 2 of them might be interesting: uvloop
and curl_cffi
(used in async mode).
The most mysterious part is that 1-2 hours before crash program started behave very very strange. Logs started to be printed in spurts, network connections started being disconnected because of timeouts. I looked into htop and discovered that process had 0.0% CPU usage in time when no logs have been printed, then wave of text printed and all over again. Looks very like program sleeps or waits for some IO synchronously, but nothing in program does it. Also it's the second or third time this happens, most of the time program works normally.
Machine and process had enough free CPU and RAM available to use so it's not resource problem. SSD and network also wasn't busy so it's definitely not resource problem.
I've been lucky to catch this using strace in htop. I didn't find a way to dump to file but here are screenshots of last lines and last lines filtered with 119
(fd that caused the issue).


CPython versions tested on:
3.13
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.13.2 (main, Feb 25 2025, 05:25:21) [GCC 12.2.0]