-
Notifications
You must be signed in to change notification settings - Fork 10
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Please confirm the following
- I agree to follow this project's code of conduct.
- I have checked the current issues for duplicates.
- I understand that dispatcherd is open source software provided for free and that I might not receive a timely response.
Bug Summary
This is really an AWX issue, but tied into this.
The assumptions that dispatcherd signal handling currently make are that the main process gets a SIGTERM and that the child processes don't. From logs using AWX, and prior assumptions about supervisord, I am fairly sure that all processes receive a SIGTERM to shut down. This leads to the following logs:
tools_awx_1 | 2025-04-19 03:21:55,184 ERROR [-] dispatcherd.service.pool Worker 4 pid=1868 failed to send exit message in 3 seconds
tools_awx_1 | 2025-04-19 03:21:55,185 ERROR [-] dispatcherd.service.pool Worker 5 pid=2068 failed to send exit message in 3 seconds
tools_awx_1 | 2025-04-19 03:21:55,185 ERROR [-] dispatcherd.service.pool Worker 6 pid=2704 failed to send exit message in 3 seconds
tools_awx_1 | 2025-04-19 03:21:55,185 ERROR [-] dispatcherd.service.pool Worker 7 pid=2707 failed to send exit message in 3 seconds
Looking up the default behavior of SIGTERM in python, it is expected that execution just stops. It seems that's what we're seeing.
It may or may not be worth intercepting signals in the child processes, but the parent process should check the child exit code as a special case in the shutdown logic.
if proc.poll() is not None:
proc.returncode # do something with this infoEnvironment
N/A
Steps to reproduce
N/A
Actual results
a
Expected results
see above
Additional information
No response
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working