Clarify Docs About Daemon Threads #125857
Description
Feature or enhancement
Ideally, we would get rid of daemon threads (as a feature) eventually. See my DPO thread. In the meantime, it makes sense (to me) to at least clarify the docs about daemon threads and steer users away from using them.
That includes the following:
- clearly identify the case(s) where daemon threads might be a suitable solution
- explicitly point out that daemon threads do not detach at exit, nor cause the process to move into the background at exit
- a warning note indicating that daemon threads should be avoided
- a brief example of using non-daemon threads to accomplish the same thing
- (maybe) a "soft deprecation" of daemon threads
existing docs: https://docs.python.org/3/library/threading.html#thread-objects
Really the only case where daemon threads might be suitable is where:
- the target comes from an extension module
- it wraps a long-running call to a third-party library
- that call does not support being interrupted or stopped
- it does not support a timeout or running for a short time
Otherwise:
- the runtime can already interrupt Python code at shutdown
- if the user maintains the long-running function then they can make it interruptible
- if it is short-running (or can be called that way) in a loop then the user only needs to add a check for exit to each iteration
- if it supports timeouts then it can be likewise put in a loop with a check each iteration
- if it supports interruption/stopping then that can be triggered at exit
Examples
Note that many of these examples make use of threading._register_atexit(), which currently isn't public API, nor documented.
Short-running task in a loop
The code for a function that supports timeouts is essentially the same.
def background_task(task):
stop = False
def atexit():
nonlocal stop
stop = True
threading._register_atexit(atexit) # currently not public API
def wrapper(*args, **kwargs):
while not stop:
task(*args, **kwargs)
return wrapper
t = threading.Thread(target=background_task(mytask))
t.start()
# Implemented as a class:
class BackgroundTask(threading.Thread):
def __init__(self, group=None, target=None, name=None, args=(), kwargs={}):
super().__init__(group, target, name, args, kwargs)
self._stop = False
def stop(self):
self._stop = True
def run(self):
threading._register_atexit(self.stop) # currently not public API
while not self._stop:
self.target(*self.args, **self.kwargs)
t = BackgroundTask(target=mytask)
t.start()Task that supports interruption or stopping
t = threading.Thread(target=mytask.run)
t.start()
threading._register_atexit(mytask.stop)Long-running Python func from a third-party package
set_async_exc = ctypes.pythonapi.PyThreadState_SetAsyncExc
set_async_exc.argtypes = (ctypes.c_ulong, ctypes.py_object)
exc = ctypes.py_object(SystemExit)
def stop_thread(tid):
# PyThreadState_SetAsyncExc(t.id, SystemExit)
set_async_exc(tid, exc)
t = threading.Thread(target=mytask)
t.start()
threading._register_atexit(stop_thread, t.ident)
# Implemented as a class:
class BackgroundTask(threading.Thread):
_set_async_exc = ctypes.pythonapi.PyThreadState_SetAsyncExc
_set_async_exc.argtypes = (ctypes.c_ulong, ctypes.py_object)
_exc = ctypes.py_object(SystemExit)
def __init__(self, group=None, target=None, name=None, args=(), kwargs={}):
super().__init__(group, target, name, args, kwargs)
def stop(self):
# PyThreadState_SetAsyncExc(t.id, SystemExit)
self._set_async_exc(self.ident, self._exc)
def run(self):
threading._register_atexit(self.stop)
super().run()
t = BackgroundTask(target=mytask)
t.start()Long-running, uninterruptible, third-party task
About your only option is something like signal.pthread_kill():
t = threading.Thread(target=mytask)
t.start()
def stop():
while t.is_alive():
try:
signal.pthread_kill(t.ident, signal.SIG_INT)
t.join(0.1)
except KeyboardInterrupt:
pass
threading._register_atexit(stop)