Unanble access GPU from container

[heavyblack1]

I have podman version 4.6.2 AMD Radeon RX 6800 XT Ubuntu 22.04 lts and my ollama I used model phi3 and doesn't have access to GPU for some reason but it should. Please can you help me? Here is how I dit it:

podman run -d --pod ollama-stack --device /dev/kfd --device /dev/dri \
    --group-add="video" \
    --security-opt unmask=/sys/dev \
    -e 'HSA_OVERRIDE_GFX_VERSION="10.3.0"' \
    -v ollama:/root/.ollama \
    --name ollama ollama/ollama:rocm 

I tried even use --privileged=false --cap-add=sys_admin didn't help or podman run -it --device /dev/kfd --device /dev/dri --net=host --security-opt=no-new-privileges --cap-drop=ALL docker.io/rocm didn't helped also.

Log:

rocBLAS error: Could not initialize Tensile host: No devices found
�time=2024-08-17T16:04:13.662Z level=ERROR source=sched.go:451 msg="error loading llama server" error="llama runner process has terminated: error:Could not initialize Tensile host: No devices found"
[GIN] 2024/08/17 - 16:04:13 | 500 |  271.603961ms |       127.0.0.1 | POST     "/api/chat"
�time=2024-08-17T16:04:18.663Z level=WARN source=sched.go:642 msg="gpu VRAM usage didn't recover within timeout" seconds=5.001198939 model=/root/.ollama/models/blobs/sha256-633fc5be925f9a484b61d6f9b9a78021eeb462100bd557309f01ba84cac26adf
�time=2024-08-17T16:04:18.913Z level=WARN source=sched.go:642 msg="gpu VRAM usage didn't recover within timeout" seconds=5.251150127 model=/root/.ollama/models/blobs/sha256-633fc5be925f9a484b61d6f9b9a78021eeb462100bd557309f01ba84cac26adf
�time=2024-08-17T16:04:19.163Z level=WARN source=sched.go:642 msg="gpu VRAM usage didn't recover within timeout" seconds=5.501029985 model=/root/.ollama/models/blobs/sha256-633fc5be925f9a484b61d6f9b9a78021eeb462100bd557309f01ba84cac26adf

[heavyblack1]

I discovered you have to use --group-add keep-groups. --group-add="video" will give you video group of container not system video group but ollama still give me same error i was able to run pytorch in container but not ollama i don't know where is problem.Its probably AMD rocm issue I will test it with rootless docker and probably report it on rocm GitHub.