Why is my GPU not being used?

[Cassiusor]

This is my first time using a Local AI and Alpaca. Everything is working fine except I have one question. My GPU is not being used at all when the AI is giving me a output. Typically local AI models use the GPU for most of the processing so I'm a bit confused why it's only using my CPU. Why is that? I am running Fedora Silverblue, Alpaca Flatpak in the Gnome Store, and a AMD GPU if that helps.

[Jeffser]

Alpaca doesn't have full support for AMD GPUs due to the flatpak sandbox, if you need to run Ollama with GPU I recommend you run Ollama on a docker and just connect it to Alpaca using the 'connect to remote instance' feature available on the preferences dialog

[Cassiusor]

Will that work on a immutable distro like Fedora Silverblue?

[Jeffser]

Fedora Silverblue comes with Podman instead of docker, let me see if I can get a command for you to run

[Cassiusor]

Ok thanks!

[Jeffser]

Alright this command is going to create a folder named Ollama wherever you run it, you can't move that folder so please choose wisely where you want Ollama to save the models

podman run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

I also recommend you this app called Pods to manage podman containers, they can be useful when using a system like silverblue