-
Notifications
You must be signed in to change notification settings - Fork 119
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem? Please describe.
GuideLLM defaults to fork as the multiprocessing context for workers. This is considered unsafe for Python applications that contain threads.
Describe the solution you'd like
We should switch to forkserver as the default multiprocessing method; as of Python 3.14 it is the default for MacOS and Linux.
Additional context
There are a few issues that must be addressed before the default can be changed:
- Most multi-processing primitives are created in the default context rather then respecting
GUIDELLM__MP_CONTEXT_TYPE. - GuideLLM is designed to explicitly import itself and all dependencies on import. This leads to slow process startups on Linux; due mostly to
transformersandtorch
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request