Skip to content

Commit cfc8482

Browse files
gloursaevesdocker
andauthored
review Compose model-runner page to use models top level element (#22963)
<!--Delete sections as needed --> ## Description Update Compose Model Runner page to use top level element `models` ## Related issues or tickets N/A ## Reviews <!-- Notes for reviewers here --> <!-- List applicable reviews (optionally @tag reviewers) --> - [ ] Technical review - [x] Editorial review - [ ] Product review --------- Signed-off-by: Guillaume Lours <[email protected]> Co-authored-by: aevesdocker <[email protected]>
1 parent d22f7c2 commit cfc8482

File tree

7 files changed

+124
-71
lines changed

7 files changed

+124
-71
lines changed
Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
---
2+
title: Use Docker Model Runner
3+
description: Learn how to integrate Docker Model Runner with Docker Compose to build AI-powered applications
4+
keywords: compose, docker compose, model runner, ai, llm, artificial intelligence, machine learning
5+
weight: 20
6+
aliases:
7+
- /compose/how-tos/model-runner/
8+
params:
9+
sidebar:
10+
badge:
11+
color: green
12+
text: New
13+
---
14+
15+
{{< summary-bar feature_name="Compose model runner" >}}
16+
17+
Docker Model Runner can be integrated with Docker Compose to run AI models as part of your multi-container applications.
18+
This lets you define and run AI-powered applications alongside your other services.
19+
20+
## Prerequisites
21+
22+
- Docker Compose v2.38 or later
23+
- Docker Desktop 4.43 or later
24+
- Docker Desktop for Mac with Apple Silicon or Docker Desktop for Windows with NVIDIA GPU
25+
- [Docker Model Runner enabled in Docker Desktop](/manuals/ai/model-runner.md#enable-docker-model-runner)
26+
27+
## Use `models` definition
28+
29+
The [`models` top-level element](/manuals/ai/compose/models-and-compose.md) in the Compose file lets you define AI models to be used by your application.
30+
Compose can then use Docker Model Runner as the model runtime.
31+
32+
The following example shows how to provide the minimal configuration to use a model within your Compose application:
33+
34+
```yaml
35+
services:
36+
my-chat-app:
37+
image: my-chat-app
38+
models:
39+
- smollm2
40+
41+
models:
42+
smollm2:
43+
image: ai/smollm2
44+
```
45+
46+
### How it works
47+
48+
During the `docker compose up` process, Docker Model Runner automatically pulls and runs the specified model.
49+
It also sends Compose the model tag name and the URL to access the model runner.
50+
51+
This information is then passed to services which declare a dependency on the model provider.
52+
In the example above, the `my-chat-app` service receives 2 environment variables prefixed by the service name:
53+
- `SMOLLM2_ENDPOINT` with the URL to access the model
54+
- `SMOLLM2_MODEL` with the model name
55+
56+
This lets the `my-chat-app` service to interact with the model and use it for its own purposes.
57+
58+
### Customizing environment variables
59+
60+
You can customize the environment variable names which will be passed to your service container using the long syntax:
61+
62+
```yaml
63+
services:
64+
my-chat-app:
65+
image: my-chat-app
66+
models:
67+
smollm2:
68+
endpoint_var: AI_MODEL_URL
69+
model_var: AI_MODEL_NAME
70+
71+
models:
72+
smollm2:
73+
image: ai/smollm2
74+
```
75+
76+
With this configuration, your `my-chat-app` service will receive:
77+
- `AI_MODEL_URL` with the URL to access the model
78+
- `AI_MODEL_NAME` with the model name
79+
80+
This allows you to use more descriptive variable names that match your application's expectations.
81+
82+
83+
## Alternative configuration with Provider services
84+
85+
> [!TIP]
86+
>
87+
> Use the []`models` top-level element](#use-models-definition) instead.
88+
89+
Compose introduced a new service type called `provider` that allows you to declare platform capabilities required by your application. For AI models, you can use the `model` type to declare model dependencies.
90+
91+
Here's an example of how to define a model provider:
92+
93+
```yaml
94+
services:
95+
chat:
96+
image: my-chat-app
97+
depends_on:
98+
- ai_runner
99+
100+
ai_runner:
101+
provider:
102+
type: model
103+
options:
104+
model: ai/smollm2
105+
```
106+
107+
Notice the dedicated `provider` attribute in the `ai_runner` service.
108+
This attribute specifies that the service is a model provider and lets you define options such as the name of the model to be used.
109+
110+
There is also a `depends_on` attribute in the `my-chat-app` service.
111+
This attribute specifies that the `my-chat-app` service depends on the `ai_runner` service.
112+
This means that the `ai_runner` service will be started before the `my-chat-app` service to allow injection of model information to the `my-chat-app` service.
113+
114+
## Reference
115+
116+
- [Docker Model Runner documentation](/manuals/ai/model-runner.md)

content/manuals/ai/compose/models-and-compose.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,4 +181,4 @@ Cloud providers might:
181181
- [`models` top-level element](/reference/compose-file/models.md)
182182
- [`models` attribute](/reference/compose-file/services.md#models)
183183
- [Docker Model Runner documentation](/manuals/ai/model-runner.md)
184-
- [Compose Model Runner documentation](/manuals/compose/how-tos/model-runner.md)]
184+
- [Compose Model Runner documentation](/manuals/ai/compose/model-runner.md)

content/manuals/ai/model-runner/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ Models are pulled from Docker Hub the first time they're used and stored locally
8484
> Using Testcontainers or Docker Compose?
8585
> [Testcontainers for Java](https://java.testcontainers.org/modules/docker_model_runner/)
8686
> and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/), and
87-
> [Docker Compose](/manuals/compose/how-tos/model-runner.md) now support Docker Model Runner.
87+
> [Docker Compose](/manuals/ai/compose/model-runner.md) now support Docker Model Runner.
8888
8989
## Enable Docker Model Runner
9090

content/manuals/compose/how-tos/model-runner.md

Lines changed: 0 additions & 66 deletions
This file was deleted.

content/manuals/compose/how-tos/provider-services.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ params:
1515
Docker Compose supports provider services, which allow integration with services whose lifecycles are managed by third-party components rather than by Compose itself.
1616
This feature enables you to define and utilize platform-specific services without the need for manual setup or direct lifecycle management.
1717

18-
1918
## What are provider services?
2019

2120
Provider services are a special type of service in Compose that represents platform capabilities rather than containers.
@@ -104,6 +103,10 @@ The plugin or binary is responsible for:
104103

105104
This information is then passed to dependent services as environment variables.
106105

106+
> [!TIP]
107+
>
108+
> If you're working with AI models in Compose, use the [`models` top-level element](/manuals/ai/compose/models-and-compose.md) instead.
109+
107110
## Benefits of using provider services
108111

109112
Using provider services in your Compose applications offers several benefits:

content/manuals/desktop/release-notes.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -219,7 +219,7 @@ For more frequently asked questions, see the [FAQs](/manuals/desktop/troubleshoo
219219
- Docker Model Runner is now available on x86 Windows machines with NVIDIA GPUs.
220220
- You can now [push models](/manuals/ai/model-runner.md#push-a-model-to-docker-hub) to Docker Hub with Docker Model Runner.
221221
- Added support for Docker Model Runner's model management and chat interface in Docker Desktop for Mac and Windows (on hardware supporting Docker Model Runner). Users can now view, interact with, and manage local AI models through a new dedicated interface.
222-
- [Docker Compose](/manuals/compose/how-tos/model-runner.md) and Testcontainers [Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner.
222+
- [Docker Compose](/manuals/ai/compose/model-runner.md) and Testcontainers [Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner.
223223
- Introducing Docker Desktop in the [Microsoft App Store](https://apps.microsoft.com/detail/xp8cbj40xlbwkx?hl=en-GB&gl=GB).
224224

225225
### Upgrades

data/summary.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ Compose menu:
110110
Compose models:
111111
requires: Docker Compose [2.38.0](/manuals/compose/releases/release-notes.md#2380) and later
112112
Compose model runner:
113-
requires: Docker Compose [2.35.0](/manuals/compose/releases/release-notes.md#2350) and later, and Docker Desktop 4.41 and later
113+
requires: Docker Compose [2.38.0](/manuals/compose/releases/release-notes.md#2350) and later, and Docker Desktop 4.43 and later
114114
Compose OCI artifact:
115115
requires: Docker Compose [2.34.0](/manuals/compose/releases/release-notes.md#2340) and later
116116
Compose provider services:

0 commit comments

Comments
 (0)