Skip to content

Commit f4c1f04

Browse files
committed
Fix broken check for insufficient GPUs
Thanks to @tsjohnso for catching!
1 parent 2e37745 commit f4c1f04

File tree

1 file changed

+2
-3
lines changed
  • server/text_generation_server/inference_engine

1 file changed

+2
-3
lines changed

server/text_generation_server/inference_engine/engine.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,8 @@ def __init__(self, model_path: str, model_config: Optional[Any]) -> None:
2222
if torch.cuda.is_available():
2323
gpu_count = torch.cuda.device_count()
2424
assert (
25-
self.world_size <= gpu_count,
26-
f"{self.world_size} shards configured but only {gpu_count} GPUs detected"
27-
)
25+
self.world_size <= gpu_count
26+
), f"{self.world_size} shards configured but only {gpu_count} GPUs detected"
2827
device_index = self.rank % torch.cuda.device_count()
2928
torch.cuda.set_device(device_index)
3029
self.device = torch.device("cuda", device_index)

0 commit comments

Comments
 (0)