Using ZLUDA with koboldcpp

[brknsoul]

Would this be possible?

ZLUDA is a CUDA wrapper for AMD GPUs.

lshyqqtiger has got ZLUDA to work with various Stable Diffusion front ends (A1111, Forge, SD.Next).

[LostRuins]

I don't have an AMD GPU. I'm not sure if the current GGML implementation using HIPBLAS works for stable diffusion, maybe you can check with @YellowRoseCx

[EmanuelOverride]

I got it to work after a bit of faff. YellowRoseCx#34 (reply in thread)

It looks like it could be added as an option without too much hassle, although it wouldn't run out of the box without installing HIP/Zluda/adding paths etc. But, if I can get it working, it's basically free.

[LioanZluda]

how did you get it to work?

[brknsoul]

Don't bother. Either use the ROCm fork, or use the no-cuda release with Vulkan backend.