-
|
Hello, I moved from nvidia to team red and I'm getting all sorts of issues. Koboldcpp loads the layers into VRAM but barely utilizes the GPU. The card is detected in the gui, but there seems to be some issue with detecting the VRAM and the capabilities of the card. Using Vulkan preset from cublas and 9070xt. Could this be a compatibility issue with a brand new card or is it something else? Any advice is greatly appreciated, ty! |
Beta Was this translation helpful? Give feedback.
Answered by
ijRadu
Mar 12, 2025
Replies: 1 comment
-
|
Turns out, flash attention was enabled when it doesn't exist on vulkan. turning it off resolved the issue |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
ijRadu
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Turns out, flash attention was enabled when it doesn't exist on vulkan. turning it off resolved the issue