@@ -21,3 +21,93 @@ Afterwards, we can browse Open WebUI on `http://127.0.0.1:8080` and register our
2121> [ !TIP]
2222>
2323> You likely want to pass a GPU into the Ollama container. Please read [ this] ( https://hub.docker.com/r/ollama/ollama ) .
24+
25+
26+ ## Windows WSL 2 Setup
27+
28+ 1 . Enable ` Nested/Hardware Virtualization ` in your BIOS
29+ 2 . Install Nvidia drivers for Windows
30+ 3 . Install Docker Desktop for Windows
31+
32+ Then open an administrative PowerShell and proceed as follows:
33+
34+ ````
35+ # install wsl
36+ wsl --install
37+
38+ # configure wsl 2 as default
39+ wsl --set-default-version 2
40+
41+ # install ubuntu
42+ wsl --install -d ubuntu
43+ ````
44+
45+ After Ubuntu installation, you are prompted for a ` username ` and ` password ` .
46+
47+ Afterwards, you will find youself in a Ubuntu terminal.
48+
49+ ## Windows Docker Desktop Setup
50+
51+ Open Docker Desktop and enable WSL 2 via the setting ` Use WSL 2 based engine ` :
52+
53+ ![ image] ( https://github.com/user-attachments/assets/85d1270a-80f9-4801-b388-81372af19f43 )
54+
55+ Then open ` Resources > WSL Integration ` and enable Docker for your Ubuntu WSL VM(s):
56+
57+ ![ image] ( https://github.com/user-attachments/assets/5b3c7ff2-d865-4e43-b867-cc931516c5d9 )
58+
59+ ## WSL 2 Ubuntu Setup
60+
61+ After Docker Desktop adjustments, return to the PowerShell with your Ubuntu terminal.
62+
63+ Proceed in Ubuntu terminal with the following commands from [ Ollama documentation] ( https://hub.docker.com/r/ollama/ollama ) :
64+
65+ ````
66+ # add repo
67+ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
68+ | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
69+ curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
70+ | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
71+ | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
72+ sudo apt-get update
73+
74+ # install nvidia container toolkit
75+ sudo apt-get install -y nvidia-container-toolkit
76+
77+ # add your user to docker group
78+ sudo usermod -aG docker <your-username>
79+ newgrp docker
80+
81+ # test docker and compose
82+ docker -v
83+ docker compose -v
84+ ````
85+
86+ > [ !WARNING]
87+ > You likely have to restart Docker Desktop and re-open the Ubuntu WSL VM
88+
89+ Now you can make use of Docker in your WSL 2 Ubuntu VM.
90+
91+ 1 . Store the ` docker-compose.yml ` from this repo into your WSL 2 Ubuntu VM
92+ 2 . Start the Docker stack consisting of Ollama and OpenWebUI
93+
94+ ````
95+ docker compose up -d
96+ ````
97+
98+ Afterwards, you can open a web browser of your choice on your Windows and browse ` http://127.0.0.1:8080 ` .
99+
100+ You will be greeted by OpenWebUI and must setup your administrative user account.
101+
102+ Afterwards:
103+ 1 . Browse the settings at ` http://127.0.0.1:8080/admin/settings `
104+ 2 . Visit the ` Connections ` area
105+ 3 . Disable ` OpenAI API `
106+ 4 . Click the ` Manage ` download button at Ollama and download your preferred models (see https://ollama.com/library )
107+
108+ ![ image] ( https://github.com/user-attachments/assets/a3b03885-3605-439e-9536-82f1933aa5b1 )
109+
110+ Unsure which Deepseek R1 model your GPU can run? Read this [ Reddit post] ( https://www.reddit.com/r/LocalLLaMA/comments/1i5wahr/comment/m8wi5tl/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button ) .
111+
112+ Here my results on Windows 11 with RTX 3080 Ti 12GB VRAM
113+ ![ image] ( https://github.com/user-attachments/assets/e8886bfd-65f8-498d-b0e0-db68700352a5 )
0 commit comments