You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/manuals/ai/model-runner.md
+36-4Lines changed: 36 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ params:
8
8
group: AI
9
9
weight: 20
10
10
description: Learn how to use Docker Model Runner to manage and run AI models.
11
-
keywords: Docker, ai, model runner, docker deskotp, llm
11
+
keywords: Docker, ai, model runner, docker desktop, docker engine, llm
12
12
aliases:
13
13
- /desktop/features/model-runner/
14
14
- /ai/model-runner/
@@ -32,6 +32,8 @@ Models are pulled from Docker Hub the first time they're used and stored locally
32
32
33
33
## Enable Docker Model Runner
34
34
35
+
### Enable DMR in Docker Desktop
36
+
35
37
1. Navigate to the **Features in development** tab in settings.
36
38
2. Under the **Experimental features** tab, select **Access experimental features**.
37
39
3. Select **Apply and restart**.
@@ -42,6 +44,36 @@ Models are pulled from Docker Hub the first time they're used and stored locally
42
44
43
45
You can now use the `docker model` command in the CLI and view and interact with your local models in the **Models** tab in the Docker Desktop Dashboard.
44
46
47
+
### Enable DMR in Docker Engine
48
+
49
+
1. Ensure you have installed [Docker Engine](/engine/install/).
50
+
2. DMR is available as a package. To install it, run:
51
+
52
+
{{< tabs >}}
53
+
{{< tab name="Ubuntu/Debian">}}
54
+
55
+
```console
56
+
$ sudo apt-get update
57
+
$ sudo apt-get install docker-model-runner
58
+
```
59
+
60
+
{{< /tab >}}
61
+
{{< tab name="RPM-base distributions">}}
62
+
63
+
```console
64
+
$ sudo yum update
65
+
$ sudo yum install docker-model-runner
66
+
```
67
+
68
+
{{< /tab >}}
69
+
{{< /tabs >}}
70
+
71
+
3. Test the installation:
72
+
73
+
```console
74
+
$ docker model status
75
+
```
76
+
45
77
## Available commands
46
78
47
79
### Model runner status
@@ -233,7 +265,7 @@ If you want to try an existing GenAI application, follow these instructions.
233
265
234
266
4. Open you app in the browser at the addresses specified in the repository [README](https://github.com/docker/hello-genai).
235
267
236
-
You'll see the GenAI app's interface where you can start typing your prompts.
268
+
You'll see the GenAI app's interface where you can start typing your prompts.
237
269
238
270
You can now interact with your own GenAI app, powered by a local model. Try a few prompts and notice how fast the responses are — all running on your machine with Docker.
239
271
@@ -378,15 +410,15 @@ Once linked, re-run the command.
378
410
379
411
### No safeguard for running oversized models
380
412
381
-
Currently, Docker Model Runner doesn't include safeguards to prevent you from launching models that exceed their system's available resources. Attempting to run a model that is too large for the host machine may result in severe slowdowns or render the system temporarily unusable. This issue is particularly common when running LLMs models without sufficient GPU memory or system RAM.
413
+
Currently, Docker Model Runner doesn't include safeguards to prevent you from launching models that exceed their system’s available resources. Attempting to run a model that is too large for the host machine may result in severe slowdowns or render the system temporarily unusable. This issue is particularly common when running LLMs models without sufficient GPU memory or system RAM.
382
414
383
415
### No consistent digest support in Model CLI
384
416
385
417
The Docker Model CLI currently lacks consistent support for specifying models by image digest. As a temporary workaround, you should refer to models by name instead of digest.
386
418
387
419
## Share feedback
388
420
389
-
Thanks for trying out Docker Model Runner. Give feedback or report any bugs you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
421
+
Thanks for trying out Docker Model Runner. Give feedback or report any bugs you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
0 commit comments