lmdeploy支持DP吗？
#2780
Replies: 1 comment 3 replies
-
|
现在需要起8个独立的服务 |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
有支持DP的打算吗？ |
Beta Was this translation helpful? Give feedback.
-
|
如果不要求服务的话，可以自行写一段代码调用 pipeline 接口。然后用 torchrun 实现 dp |
Beta Was this translation helpful? Give feedback.
-
|
这样可行，但是感觉不太优雅。 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
8张A100用于7B推理，只能起8个独立的服务吗还是支持DP=8？
Beta Was this translation helpful? Give feedback.
All reactions