Skip to content

Add a warning for special devices #15563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Aug 26, 2025
Merged

Add a warning for special devices #15563

merged 5 commits into from
Aug 26, 2025
+21 −1

Conversation

pt13762104
Copy link
Contributor

Add a warning for special GTX series cards that have low performance due to MMA kernels.

@github-actions github-actions bot added Nvidia GPU Issues specific to Nvidia GPUs ggml changes relating to the ggml tensor library for machine learning labels Aug 25, 2025
JohannesGaessler
JohannesGaessler reviewed Aug 25, 2025
View reviewed changes
Copy link
Collaborator

@JohannesGaessler JohannesGaessler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly just minor changes. I've revised the recommended compilation options since it's still possible to use GTX 16XX in conjunction with Ampere or newer.

@pt13762104 @JohannesGaessler
Apply suggestions from code review
a252cf0 
Co-authored-by: Johannes Gäßler <[email protected]>
JohannesGaessler
JohannesGaessler reviewed Aug 25, 2025
View reviewed changes
Copy link
Collaborator

@JohannesGaessler JohannesGaessler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the renaming of the vector you can't just accept the suggested change on Github because it needs to be changed consistently. If you fix that I'll merge this PR.

@pt13762104
Copy link
Contributor Author

pt13762104 commented Aug 25, 2025 •
edited
Loading

It now compiles and runs successfully.

@JohannesGaessler JohannesGaessler merged commit 4c37636 into ggml-org:master Aug 26, 2025
48 checks passed
@CISC
Copy link
Collaborator

CISC commented Aug 26, 2025

This causes an abort unless you have compiled in turing support, see #15584

@pt13762104
Copy link
Contributor Author

pt13762104 commented Aug 26, 2025 •
edited
Loading

I would guess that's due to 7d3e9fd#diff-cb2761994492f7839320b81765a6fc2a23c180e3ddfd4a51d6b0f618f8d76a69R277 ? A fix might be on the lines of replacing ggml_cuda_highest_compiled_arch to ggml_cuda_has_arch, something like
pt13762104/llama.cpp@f671559. Could anyone test if this works?

@XZVB12
Copy link

XZVB12 commented Aug 26, 2025 •
edited
Loading

I would guess that's due to 7d3e9fd#diff-cb2761994492f7839320b81765a6fc2a23c180e3ddfd4a51d6b0f618f8d76a69R277 ? A fix might be on the lines of replacing ggml_cuda_highest_compiled_arch to ggml_cuda_has_arch, something like pt13762104@f671559. Could anyone test if this works?

seems to work, at least the model was successfully loaded and answer generated

@XZVB12 XZVB12 mentioned this pull request Aug 26, 2025
Closed
@blueyred
Copy link

This commit seems to break llama-server on my system:
#15593

@JohannesGaessler
Copy link
Collaborator

Should be fixed with #15587 .

Minh141120 pushed a commit to menloresearch/llama.cpp that referenced this pull request Aug 27, 2025
@pt13762104 @JohannesGaessler @Minh141120
Add a warning for special devices (ggml-org#15563)
830e2f4 
* Add warning

* Print the devices names

* Add newlines

* Apply suggestions from code review

Co-authored-by: Johannes Gäßler <[email protected]>

* Fix vector names

---------

Co-authored-by: Johannes Gäßler <[email protected]>
Nexesenex added a commit to Nexesenex/croco.cpp that referenced this pull request Oct 6, 2025
@Nexesenex
Revert "Add a warning for special devices (ggml-org#15563)"
7ce6205 
This reverts commit 4c37636.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JohannesGaessler JohannesGaessler JohannesGaessler approved these changes

Assignees

No one assigned

Labels

ggml changes relating to the ggml tensor library for machine learning Nvidia GPU Issues specific to Nvidia GPUs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@pt13762104 @CISC @XZVB12 @blueyred @JohannesGaessler