Attempting to connect to upstream llama-server ui via llama-swap results in a 404

[doctorjames]

Describe the bug
I have just updated the llama-swap docker container to the latest cuda version (v159-cuda-b6503) and the upstream web UI no longer works. This is likely caused by the new svelte web UI in llama-server.

Looking at the logs, included below, and confirmed in the Firefox debugger, it looks like the new svelte app tried to load an absolute path from the server. So this might be an issue that needs coordinating with the upstream llama-server folks.

Seems related: ggml-org/llama.cpp#16079

Expected behaviour
Upstream web interface is available.

Operating system and version

• OS: Linux/docker v159-cuda-b6503

Proxy Logs

[INFO] Request 172.25.250.1 "GET /upstream/qwen3-4b/ HTTP/1.1" 200 842295 "Mozilla/5.0 (X11; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0" 2.866833ms
[INFO] Request 172.25.250.1 "GET /_app/version.json HTTP/1.1" 404 0 "Mozilla/5.0 (X11; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0" 6.208µs
[INFO] Request 172.25.250.1 "GET /props HTTP/1.1" 404 0 "Mozilla/5.0 (X11; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0" 5.666µs

[Crandel]

I can confirm the same behavior after latest llama.cpp update. When I directly connected to llama.cpp server I've noticed they completely refactor their frontend, so maybe this could be the issue?

[repsac-by]

Related to sveltejs/kit#9569

[mostlygeek]

This change likely broke many llama-server use cases where it is behind a proxy and not served on /.

The previous UI used relative paths which is the right approach IMO.

Workarounds for now would be:

• to downgrade to a container with an older version of llama-server.
• use the old ui, see this comment

[notgood]

Confirming error 404 since switch to sveltejs.

Perhaps direct link to currently running model webui could be provided somewhere in "Upstream Logs" section.

[ServeurpersoCom]

I'm in the case. To keep llama-swap statistics i use a main reverse proxy for all and I add a model selector (proposal ?) :)

video.mp4

Also to work with proxy we need a .htaccess like :
RewriteEngine On
RewriteRule ^chat/[^/]+/?$ index.html [L]
Without it, browser F5 on a selected conversation makes 404 error on /chat URLs (Svelte virtual routes).

[antropiq]

The latest workable tag is https://github.com/ggml-org/llama.cpp/releases/tag/b6499 just before the SvelteKit-based WebUI (#14839) switch on llama.cpp side. I get that same 404 error with b6500 and newer 😕

[isaac-mcfadyen]

Looks like ggml-org/llama.cpp#16079 may help resolve this. Once it's merged maybe llama-swap could add the env var automatically?

EDIT: looks like that PR changes the base path at build time which won't work here.

[ServeurpersoCom]

Yes this is a build-time variable, not a run-time variable. The question that remains is whether it is possible to do true relative web with Svelte, I did not succeed so I made this compromise as cleanly as possible to no longer be locked at the root, which allows operation through llama-swap as a single URL reverse proxy.

Aand llama-swap might also not like Svelte routes... ( /chat/a45786bf-2f75-40b6-8ce1-032dd0f308fe )

[mostlygeek]

A simple work around is to use the old UI with llama-server. Here's how you can do it:

• Download: the old UI - index.html.gz and put it in a directory somewhere. I use /path/to/llama-cpp-static
• decompress it with: gunzip index.html.gz
• use --path /path/to/llama-cpp-static in your llama-server config

I use a macro in my config.yaml and it was as simple as adding one line to update all configurations:

macros: 
macros:
  "server-latest": |
    /path/to/llama-server/llama-server-latest
    --host 127.0.0.1 --port ${PORT}
    -ngl 999 -ngld 999
    --no-mmap
    --no-warmup

    # use previous UI until new svelte based one can support paths
    --path /path/to/llama-cpp-static/

I built the index.html.gz from the last commit to the webui before it was replaced with svelte. If you want to build it yourself you can:

• clone llama.cpp repo
• cd tools/server/webui
• npm install && npm run build
• an index.html will be in the dist/ folder

I considered workarounds like #311 but I think there are some trade offs that can be discussed in that issue.