Linux (Fedora) containers uses CPU instead of GPU (Intel Arc A770 16Gb)

[Clm-1138]

Hello,

I am looking for assistance because I don't know what I am doing wrong.

I am using the following system

OS: Fedora Linux 42 (Server Edition) x86_64
Kernel: Linux 6.15.7-200.fc42.x86_64
CPU: Intel(R) Core(TM) i5-8400T (6) @ 3.30 GHz
GPU: Intel Arc A770 @ 2.40 GHz [Discrete]
Memory: 946.14 MiB / 15.56 GiB

• Intel OneAPI is installed
• drivers looks okay (I don't know how to find newer ones for Fedora 42, if any)

I am running Local-Ai through docker with the following docker file :

services:
  api:
    image: localai/localai:v3.3.0-gpu-intel-f32
    container_name: local-ai-intel-fp32
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/readyz"]
      interval: 1m
      timeout: 20m
      retries: 5
    build:
      context: .
      dockerfile: Dockerfile
      args:
      - IMAGE_TYPE=core
      - BASE_IMAGE=ubuntu:22.04
    ports:
      - 8080:8080
    devices:
     - /dev/dri/card0
     - /dev/dri/renderD128
    env_file:
      - .env
    environment:
      - MODELS_PATH=/models
      - DEBUG=true
      - SYCL_DEVICE_FILTER=GPU
      - SYCL_BACKEND=level_zero
    volumes:
      - ./models:/models:cached
      - ./images/:/tmp/generated/images/
    command:
    - qwen3-8b
    - mistral-7b-instruct-v0.3

My GPU seems to be accessible from the container :

root@00932e8ac798:/# sycl-ls
[level_zero:gpu][level_zero:0] Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Arc(TM) A770 Graphics 12.55.8 [1.6.33276+22]
[opencl:cpu][opencl:0] Intel(R) OpenCL, Intel(R) Core(TM) i5-8400T CPU @ 1.70GHz OpenCL 3.0 (Build 0) [2025.20.6.0.04_224945]
[opencl:gpu][opencl:1] Intel(R) OpenCL Graphics, Intel(R) Arc(TM) A770 Graphics OpenCL 3.0 NEO  [25.13.33276]
[opencl:cpu][opencl:2] Intel(R) OpenCL, Intel(R) Core(TM) i5-8400T CPU @ 1.70GHz OpenCL 3.0 (Build 0) [2025.20.6.0.04_224945]

I tried the following images :

• localai/localai:v3.3.0-gpu-intel-f32
• localai/localai:master-gpu-intel-f16
• localai/localai:master-gpu-vulkan

And yet, every time, the cpu is doing all the work.
Here is something the logs keeps telling :

api-1  | 10:14AM DBG Installing backend "llama-cpp"
api-1  | 10:14AM DBG GPU vendor gpuVendor=
api-1  | 10:14AM DBG GPU vendor gpuVendor=
api-1  | 10:14AM DBG guessDefaultsFromFile: NGPULayers set NGPULayers=99999999
api-1  | 10:14AM DBG guessDefaultsFromFile: template already set name=mistral-7b-instruct-v0.3
api-1  | 10:14AM DBG guessDefaultsFromFile: NGPULayers set NGPULayers=99999999
api-1  | 10:14AM DBG guessDefaultsFromFile: template already set name=qwen3-8b
api-1  | 10:14AM DBG Registering backend name=cpu-llama-cpp runFile=/backends/cpu-llama-cpp/run.sh
api-1  | 10:14AM DBG Registering backend name=llama-cpp runFile=/backends/cpu-llama-cpp/run.sh

I really need some help here because I must be doing something wrong, but I do not know what.

[Clm-1138]

I also tried the docker run route, but without sucess :

clement@localhost:~$ sudo docker run -e DEBUG=true --privileged -ti -v $PWD/models:/models --gpus all -p 808
0:8080  -v /dev/dri:/dev/dri --rm localai/localai:master-gpu-intel-f32 qwen3-8b
Unable to find image 'localai/localai:master-gpu-intel-f32' locally
master-gpu-intel-f32: Pulling from localai/localai
Digest: sha256:91497b443fd27f0b3d07fa9ab9917b362cb44d9d6714c7faf49746add46e5666
Status: Downloaded newer image for localai/localai:master-gpu-intel-f32
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]

Yet this one looks like it's telling me something I don't understand...

[mudler]

Hello,

I am looking for assistance because I don't know what I am doing wrong.

I am using the following system

OS: Fedora Linux 42 (Server Edition) x86_64
Kernel: Linux 6.15.7-200.fc42.x86_64
CPU: Intel(R) Core(TM) i5-8400T (6) @ 3.30 GHz
GPU: Intel Arc A770 @ 2.40 GHz [Discrete]
Memory: 946.14 MiB / 15.56 GiB

* Intel OneAPI is installed

* drivers looks okay (I don't know how to find newer ones for Fedora 42, if any)

I am running Local-Ai through docker with the following docker file :

services:
  api:
    image: localai/localai:v3.3.0-gpu-intel-f32
    container_name: local-ai-intel-fp32
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/readyz"]
      interval: 1m
      timeout: 20m
      retries: 5
    build:
      context: .
      dockerfile: Dockerfile
      args:
      - IMAGE_TYPE=core
      - BASE_IMAGE=ubuntu:22.04
    ports:
      - 8080:8080
    devices:
     - /dev/dri/card0
     - /dev/dri/renderD128
    env_file:
      - .env
    environment:
      - MODELS_PATH=/models
      - DEBUG=true
      - SYCL_DEVICE_FILTER=GPU
      - SYCL_BACKEND=level_zero
    volumes:
      - ./models:/models:cached
      - ./images/:/tmp/generated/images/
    command:
    - qwen3-8b
    - mistral-7b-instruct-v0.3

My GPU seems to be accessible from the container :

root@00932e8ac798:/# sycl-ls
[level_zero:gpu][level_zero:0] Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Arc(TM) A770 Graphics 12.55.8 [1.6.33276+22]
[opencl:cpu][opencl:0] Intel(R) OpenCL, Intel(R) Core(TM) i5-8400T CPU @ 1.70GHz OpenCL 3.0 (Build 0) [2025.20.6.0.04_224945]
[opencl:gpu][opencl:1] Intel(R) OpenCL Graphics, Intel(R) Arc(TM) A770 Graphics OpenCL 3.0 NEO  [25.13.33276]
[opencl:cpu][opencl:2] Intel(R) OpenCL, Intel(R) Core(TM) i5-8400T CPU @ 1.70GHz OpenCL 3.0 (Build 0) [2025.20.6.0.04_224945]

I tried the following images :

* localai/localai:v3.3.0-gpu-intel-f32

* localai/localai:master-gpu-intel-f16

* localai/localai:master-gpu-vulkan

And yet, every time, the cpu is doing all the work. Here is something the logs keeps telling :

api-1  | 10:14AM DBG Installing backend "llama-cpp"
api-1  | 10:14AM DBG GPU vendor gpuVendor=
api-1  | 10:14AM DBG GPU vendor gpuVendor=
api-1  | 10:14AM DBG guessDefaultsFromFile: NGPULayers set NGPULayers=99999999
api-1  | 10:14AM DBG guessDefaultsFromFile: template already set name=mistral-7b-instruct-v0.3
api-1  | 10:14AM DBG guessDefaultsFromFile: NGPULayers set NGPULayers=99999999
api-1  | 10:14AM DBG guessDefaultsFromFile: template already set name=qwen3-8b
api-1  | 10:14AM DBG Registering backend name=cpu-llama-cpp runFile=/backends/cpu-llama-cpp/run.sh
api-1  | 10:14AM DBG Registering backend name=llama-cpp runFile=/backends/cpu-llama-cpp/run.sh

I really need some help here because I must be doing something wrong, but I do not know what.

The docker-compose file is correct. There is currently a bug in LocalAI as such the intel gpu is not identified correctly. This is being fixed in #5945, but meanwhile, as a temporary workaround you can set the LOCALAI_FORCE_META_BACKEND_CAPABILITY variable manually to intel to force LocalAI to pull the intel backends.

This is documented in https://localai.io/features/gpu-acceleration/#automatic-backend-detection .

[Clm-1138]

Thanks a lot, I'm gonna try this out as soon as possible and will keep you posted.

[Clm-1138]

Okay, update, last time, CPU was doing the heavy lifting, this time, it offloads to the GPU... which does noting.
When using intel_gpu_top there is a very short burst of activity before nothing more happens.

GPU is working. Both Unigine Heaven and FurMark are using it properly. Performance is not stellar, but it works.
I am using pcie gen 3 and have no acess to resizeableBAR if it is of some relevance...