You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+3-39Lines changed: 3 additions & 39 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,6 +41,9 @@ docker version
41
41
42
42
# Check Docker Model Runner version
43
43
docker model version
44
+
45
+
# Run a model to test the full setup
46
+
docker model run ai/gemma3 "Hello"
44
47
```
45
48
46
49
If `docker model` is not available, see the troubleshooting section below.
@@ -350,45 +353,6 @@ The response will contain the model's reply:
350
353
}
351
354
```
352
355
353
-
## NVIDIA NIM Support
354
-
355
-
Docker Model Runner supports running NVIDIA NIM (NVIDIA Inference Microservices) containers directly. This provides a simplified workflow for deploying NVIDIA's optimized inference containers.
356
-
357
-
### Prerequisites
358
-
359
-
- Docker with NVIDIA GPU support (nvidia-docker2 or Docker with NVIDIA Container Runtime)
360
-
- NGC API Key (optional, but required for some NIM models)
361
-
- Docker login to nvcr.io registry
362
-
363
-
### Quick Start
364
-
365
-
1.**Login to NVIDIA Container Registry:**
366
-
367
-
```bash
368
-
docker login nvcr.io
369
-
Username: $oauthtoken
370
-
Password: <PASTE_API_KEY_HERE>
371
-
```
372
-
373
-
2.**Set NGC API Key (if required by the model):**
374
-
375
-
```bash
376
-
export NGC_API_KEY=<PASTE_API_KEY_HERE>
377
-
```
378
-
379
-
3.**Run a NIM model:**
380
-
381
-
```bash
382
-
docker model run nvcr.io/nim/google/gemma-3-1b-it:latest
383
-
```
384
-
385
-
That's it! The Docker Model Runner will:
386
-
- Automatically detect that this is a NIM image
387
-
- Pull the NIM container image
388
-
- Configure it with proper GPU support, shared memory (16GB), and NGC credentials
389
-
- Start the container and wait for it to be ready
390
-
- Provide an interactive chat interface
391
-
392
356
### Features
393
357
394
358
-**Automatic GPU Detection**: Automatically configures NVIDIA GPU support if available
0 commit comments