changes to the manual for Self Hosted AI Starter Kit

[2morowMan]

This has taken me a week!!!!!! to figure out. And the solution is a slap in the face.

https://github.com/n8n-io/self-hosted-ai-starter-kit
. . . tells nvidia gpu users to follow Ollama Docker Instructions . . .
After going down the rabbit hole of Ollama Docker -> nVidia Contain Toolkit -> NVIDIA GPU driver for your Linux distribution . . .
discovering that method is well out of date - we need windows only drivers . . . finding nvidia cuda drivers with WSL 2 GPU Paravirtualization support in the dev areas . . . And totally ruining my pc again, and again, again . . .

i eventually found this
https://developer.nvidia.com/cuda/wsl
which takes you to the regular nvidia drivers AS THEY INCLUDE CUDA-FOR-WSL2

i dont know whether to laugh or scream. i've been down rabbit holes inside rabbit holes. My brain is fried.
i strongly suggest the instructions are updated. Maybe not say anything about nvidia vs amd gpu's?! or just a sentence saying you're ok, if youre drivers are from 2025 !! (or whenever this changed).

I also found this
https://docs.docker.com/desktop/features/gpu/
which has a wonderful command to test your setup
docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

my fresh install of everything passed. So once ive recovered i can continue...

anyone know about this? an extra sentence or two might save others HOURS of headaches.
Thanks.

[2morowMan]

And i just found out that Docker Desktop includes everything needed of the "nvidia container toolkit"
https://docs.docker.com/desktop/release-notes/#4420
https://github.com/NVIDIA/nvidia-container-toolkit/releases/tag/v1.17.7

So using Docker Desktop, and latest nvidia drivers, we users do not need to concern ourselves at all with any extra steps to fully utilise our cuda nvidia gpus.
Lucky us. So lets update the installation instructions. what say you all ?

[jeanpaul]

This is about how to run Ollama in docker, correct? If the Ollama docs are incorrect? Have you tried updating the Ollama docs first? That seems the canonical reference for getting Ollama up and running. I understand your frustration, but this is not a repository that tells you the most optimal way of running everything.

If you fail to get your changes into the Ollama docs, please provide some links to (public) conversations on it!

[2morowMan]

quote...
https://github.com/n8n-io/self-hosted-ai-starter-kit
. . . tells nvidia gpu users to follow Ollama Docker Instructions . . .

So the n8n docs need updating.

If you unnecessarily tell users to go to other sites, that have incorrect/outdated information, problems ensue.

FIX...
do NOT tell users to do extra steps if they have nVidia.