Any way to set a maximum memory limit on inference mode ? #3994
Unanswered
Mactarvish
asked this question in
Q&A
Replies: 1 comment
-
|
Why do you need to ? What are you doing exactly ? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I'm trying to running inference model on a K8S env, and I'm willing to set the maximum GPU memory limit to 8G, i.e. the rest part more than 8G shall be handled by CPU. Any way to set that? Thanks.
Beta Was this translation helpful? Give feedback.
All reactions