llama.cpp and 7G Llama-2-Chat model: resource requirements, if possible #5172
Unanswered
christian-2
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am a new to this project and would like to try inference with llama.cpp and a 7G Llama-2-Chat model: is this combination currently supported and what are the resource requirements? I have e.g. a VM with 8 vCPUs and 16 GB RAM or (if need be) a bare-medal server with 56 CPUs and 500 GB RAM available. The (underlying) hardware is fairly recent in both cases. I guess the bare-metal second could serve in principle, but could the VM as well?
Beta Was this translation helpful? Give feedback.
All reactions