关于训练，怎么模型并行？
#613
Replies: 1 comment
-
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
在训练过程中，官方给出的ptuningv2训练脚本应该是数据并行脚本，即将模型的参数复制到num_gpus张GPU上，然后在将example分配给每个GPU并行训练。
但是想问一下怎么实现模型并行+数据并行，即将模型拆分加载到多张GPU上，然后再数据并行？
Beta Was this translation helpful? Give feedback.
All reactions