Skip to content

Add option for CUDA 13.1 llama.cpp in installer#7474

Open
GodEmperor785 wants to merge 1 commit intooobabooga:devfrom
GodEmperor785:cuda13_llamacpp
Open

Add option for CUDA 13.1 llama.cpp in installer#7474
GodEmperor785 wants to merge 1 commit intooobabooga:devfrom
GodEmperor785:cuda13_llamacpp

Conversation

@GodEmperor785
Copy link
Copy Markdown
Contributor

This PR adds new choice for NVIDIA option in installer - choose CUDA version for llama.cpp. Other options are unchanged.

So now choices looks like this when user selects "A) NVIDIA" when installing they have another choice:

Which CUDA version do you want for llama.cpp?

A) CUDA 12.4 - Supports all GPUs
B) CUDA 13.1 - Faster on RTX 50xx Blackwell, but drops GTX 750/900/1000 and Titan V (needs driver >= 580)

https://github.com/oobabooga/llama-cpp-binaries already provides builds for CUDA 13.1 for llama.cpp, also portable builds provide version with CUDA 13.1.

Made this as a choice for user because some people might have old GPUs or just might want to install old builds for some reason.

For users with existing installation it doesn't reinstall anything - old installs will stay on 12.4. User can always delete installer_files and reinstall for CUDA 13.1

I verified locally that installation of CUDA 12.4/13.1 binaries works (fresh install) and that running from old install still works. Tested on Windows + Nvidia GPU.

Checklist:

Adding new choice for NVIDIA option in installer - choose CUDA version for llama.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant