Lemonade LLMs in docker container (Linux)

[VladimirVLF]

You can easily run Lemonade in docker if you are on Linux, e.g. Ubuntu.
Here Lemonade with llamacpp with Vulkan back-end is bundled with Open-WebUi into a docker compose service.
NB: It will likely not work on Windows host due to device sharing /dev/dri:/dev/dri.

You will need Docker installation and the two files below: Dockerfile and docker-compose.yml

Dockerfile (change the URL to Lemonade installer and context window size as needed):

FROM ubuntu:24.04

# Configure URL to Lemonade version, here v9.0.2
ENV LEMONADE_DEB=https://github.com/lemonade-sdk/lemonade/releases/download/v9.0.2/lemonade-server-9.0.2-Linux.deb

RUN apt-get update && \
    apt-get install -y \
        curl \
        unzip \
        libgomp1 \
        libvulkan1 \
        mesa-vulkan-drivers && \
    curl -L -o /tmp/lemonade.deb $LEMONADE_DEB && \
    apt-get install -y /tmp/lemonade.deb && \
    apt-get autoremove -y && \
    apt-get clean -y && \
    rm -rf /tmp/* /var/tmp/* && \
    find /var/cache/apt/archives /var/lib/apt/lists -not -name lock -type f -delete && \
    find /var/cache -type f -delete

VOLUME /models
VOLUME /usr/local/share/lemonade-server/llama

EXPOSE 8000

ENV HF_HOME=/models
ENV LEMONADE_HOST=0.0.0.0
ENV LEMONADE_PORT=8000
ENV LEMONADE_LLAMACPP=vulkan
# Configure context window size as needed, here 4k. For 64k: 65536. For 128k: 131072
ENV LEMONADE_CTX_SIZE=4096

ENTRYPOINT [ "lemonade-server" ]
CMD ["serve"]

docker-compose.yml (configure your path to models):

services:
  lemonade:
    image: lemonade:latest
    container_name: lemonade
    restart: unless-stopped
    ports:
      - "127.0.0.1:8000:8000"
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - lemonadellama:/usr/local/share/lemonade-server/llama
      # Configure path to your model storage, here: /mnt/data/models
      - /mnt/data/models:/models
  openwebui:
    image: ghcr.io/open-webui/open-webui:main-slim
    container_name: openwebui
    restart: unless-stopped
    ports:
      - "127.0.0.1:8080:8080"
    volumes:
      - openwebui:/app/backend/data
volumes:
  openwebui:
  lemonadellama:

To get it working follow these steps:

1. Build Lemonade docker image as lemonade:latest:

docker build -t lemonade:latest .

2. Launch the docker compose project as service, so that your chat is always available:

docker compose -p ai up -d

3. Go to Lemonad web UI at http://127.0.0.1:8000/ and download a model, let's say a small reasoning model Qwen3-0.6B-GGUF
4. Go to Open WebUI at http://127.0.0.1:8080/, create admin account and login
5. Connect Open WebUI to Lemonade: Admin panel -> Settings -> Connections -> OpenAI API connection URL: http://lemonade:8000/api/v1

Now you should be able to launch a chat in Open WebUI, select the downloaded Lemonade model and chat with it.

For reference, my laptop with Ryzen AI 7 Pro 360 with iGPU Radeon 880M and 64 GB vRAM (max 32GB for iGPU) runs gpt-oss-20b-mxfp4-GGUF model with 128k context window size at 27.8 tokens per second.
I also had amdgpu driver with xrt and xdna installed on the laptop native Ubuntu OS. Not sure if it matters.