Skip to content

Commit 86ea597

Browse files
committed
Compatible with lower versions of torch
1 parent fe315a5 commit 86ea597

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

lightllm/server/router/model_infer/mode_backend/base_backend.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,8 +84,10 @@ def init_model(self, kvargs):
8484
init_method=f'tcp://127.0.0.1:{kvargs["nccl_port"]}',
8585
rank=self.tp_rank,
8686
world_size=self.world_size,
87-
device_id=torch.device(f"cuda:{self.tp_rank}"),
8887
)
88+
# warmup nccl communicator
89+
a = torch.zeros([1]).to(f"cuda:{self.tp_rank}")
90+
dist.all_reduce(a)
8991

9092
from lightllm.distributed import set_custom_reduce
9193

0 commit comments

Comments
 (0)