Replies: 1 comment 1 reply
-
|
你可以直接 uvicorn命令行运行 这样可以开启多个wokers |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
已解决，感谢大佬 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
当workers大于1时，显示报错是模型未定义，即便将model设为全局变量也是一样。
所以想请问一下如何实现多进程呢？
还想和大家讨论一下如何在多进程调用一个llm的条件下如何节约显存的问题。
Beta Was this translation helpful? Give feedback.
All reactions