请问是否支持多级多卡 #30
lainxx
announced in
Announcements
请问是否支持多级多卡
#30
Replies: 3 comments
-
|
使用ddp_train train_pretrain_stage0.py --standalone --nnodes=4 --nproc-per-node=8试一下 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
好的我试试 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
不过还是建议使用deepspeed，使用方式smart_train xxx.py deepspeed参数 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
采用指令 torchrun --standalone --nnodes=4 --nproc-per-node=8 train_pretrain_stage0.py
报错：torch.distributed.elastic.rendezvous.api.RendezvousTimeoutError
Beta Was this translation helpful? Give feedback.
All reactions