Skip to content

Commit 9890482

Browse files
authored
Refactor LitServer initialization parameters for type safety (#555)
- Updated the `accelerator` parameter to use a more specific Literal type, allowing only "cpu", "cuda", "mps", or "auto". - Changed the `devices` parameter to accept either an integer or the Literal "auto", enhancing clarity and type checking. - Added a comment indicating that several arguments are deprecated and will be removed in version 0.3.0.
1 parent bef6616 commit 9890482

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

src/litserve/server.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -611,8 +611,8 @@ def predict(self, prompt):
611611
def __init__(
612612
self,
613613
lit_api: Union[LitAPI, List[LitAPI]],
614-
accelerator: str = "auto",
615-
devices: Union[str, int] = "auto",
614+
accelerator: Literal["cpu", "cuda", "mps", "auto"] = "auto",
615+
devices: Union[int, Literal["auto"]] = "auto",
616616
workers_per_device: int = 1,
617617
timeout: Union[float, bool] = 30,
618618
healthcheck_path: str = "/health",
@@ -627,6 +627,7 @@ def __init__(
627627
middlewares: Optional[list[Union[Callable, tuple[Callable, dict]]]] = None,
628628
loggers: Optional[Union[Logger, List[Logger]]] = None,
629629
fast_queue: bool = False,
630+
# All the following arguments are deprecated and will be removed in v0.3.0
630631
max_batch_size: Optional[int] = None,
631632
batch_timeout: float = 0.0,
632633
stream: bool = False,

0 commit comments

Comments
 (0)