You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/manuals/ai/model-runner.md
+36-1Lines changed: 36 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ params:
8
8
group: AI
9
9
weight: 20
10
10
description: Learn how to use Docker Model Runner to manage and run AI models.
11
-
keywords: Docker, ai, model runner, docker deskotp, llm
11
+
keywords: Docker, ai, model runner, docker desktop, docker engine, llm
12
12
aliases:
13
13
- /desktop/features/model-runner/
14
14
- /ai/model-runner/
@@ -45,6 +45,37 @@ Models are pulled from Docker Hub the first time they're used and stored locally
45
45
46
46
You can now use the `docker model` command in the CLI and view and interact with your local models in the **Models** tab in the Docker Desktop Dashboard.
47
47
48
+
### Enable DMR in Docker Engine
49
+
50
+
1. Ensure you have installed [Docker Engine](/engine/install/).
51
+
2. DMR is available as a package. To install it, run:
52
+
53
+
{{< tabs >}}
54
+
{{< tab name="Ubuntu/Debian">}}
55
+
56
+
```console
57
+
$ sudo apt-get update
58
+
$ sudo apt-get install docker-model-plugin
59
+
```
60
+
61
+
{{< /tab >}}
62
+
{{< tab name="RPM-base distributions">}}
63
+
64
+
```console
65
+
$ sudo dnf update
66
+
$ sudo dnf install docker-model-plugin
67
+
```
68
+
69
+
{{< /tab >}}
70
+
{{< /tabs >}}
71
+
72
+
3. Test the installation:
73
+
74
+
```console
75
+
$ docker model version
76
+
$ docker model run ai/smollm2
77
+
```
78
+
48
79
## Integrate the Docker Model Runner into your software development lifecycle
49
80
50
81
You can now start building your Generative AI application powered by the Docker Model Runner.
@@ -143,6 +174,10 @@ To call the `chat/completions` OpenAI endpoint from the host via TCP:
143
174
144
175
1. Enable the host-side TCP support from the Docker Desktop GUI, or via the [Docker Desktop CLI](/manuals/desktop/features/desktop-cli.md).
145
176
For example: `docker desktop enable model-runner --tcp <port>`.
177
+
178
+
If you are running on Windows, also enable GPU-backed inference.
179
+
See [Enable Docker Model Runner](#enable-dmr-in-docker-desktop).
180
+
146
181
2. Interact with it as documented in the previous section using `localhost` and the correct port.
Copy file name to clipboardExpand all lines: content/manuals/engine/release-notes/28.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -82,7 +82,7 @@ For a full list of pull requests and changes in this release, refer to the relev
82
82
83
83
### Networking
84
84
85
-
- Add bridge network option `"com.docker.network.bridge.trusted_host_interfaces"`, accepting a space-separated list of interface names. These interfaces have direct access to published ports on container IP addresses. [moby/moby#49832](https://github.com/moby/moby/pull/49832)
85
+
- Add bridge network option `"com.docker.network.bridge.trusted_host_interfaces"`, accepting a colon-separated list of interface names. These interfaces have direct access to published ports on container IP addresses. [moby/moby#49832](https://github.com/moby/moby/pull/49832)
86
86
- Add daemon option `"allow-direct-routing"` to disable filtering of packets from outside the host addressed directly to containers. [moby/moby#49832](https://github.com/moby/moby/pull/49832)
87
87
- Do not display network options `com.docker.network.enable_ipv4` or `com.docker.network.enable_ipv6` in inspect output if they have been overridden by `EnableIPv4` or `EnableIPv6` in the network create request. [moby/moby#49866](https://github.com/moby/moby/pull/49866)
88
88
- Fix an issue that could cause network deletion to fail after a daemon restart, with error "has active endpoints" listing empty endpoint names. [moby/moby#49901](https://github.com/moby/moby/pull/49901)
0 commit comments