Skip to content

Commit 1e8dba8

Browse files
committed
delete duplicated content
1 parent c48963b commit 1e8dba8

File tree

4 files changed

+55
-144
lines changed

4 files changed

+55
-144
lines changed

content/manuals/ai/compose/model-runner.md

Lines changed: 0 additions & 137 deletions
This file was deleted.

content/manuals/ai/compose/models-and-compose.md

Lines changed: 53 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@ title: Define AI Models in Docker Compose applications
33
linkTitle: Use AI models in Compose
44
description: Learn how to define and use AI models in Docker Compose applications using the models top-level element
55
keywords: compose, docker compose, models, ai, machine learning, cloud providers, specification
6+
alias:
7+
- /compose/how-tos/model-runner/
8+
- /ai/compose/model-runner/
69
weight: 10
710
params:
811
sidebar:
@@ -18,11 +21,16 @@ Compose lets you define AI models as core components of your application, so you
1821
## Prerequisites
1922

2023
- Docker Compose v2.38 or later
21-
- A platform that supports Compose models such as Docker Model Runner or compatible cloud providers
24+
- A platform that supports Compose models such as Docker Model Runner or compatible cloud providers.
25+
If you are using DMR:
26+
27+
- Docker Desktop 4.43 or later
28+
- Docker Desktop for Mac with Apple Silicon or Docker Desktop for Windows with NVIDIA GPU
29+
- [Docker Model Runner enabled in Docker Desktop](/manuals/ai/model-runner.md#enable-docker-model-runner)
2230

2331
## What are Compose models?
2432

25-
Compose `models` are a standardized way to define AI model dependencies in your application. By using the []`models` top-level element](/reference/compose-file/models.md) in your Compose file, you can:
33+
Compose `models` are a standardized way to define AI model dependencies in your application. By using the [`models` top-level element](/reference/compose-file/models.md) in your Compose file, you can:
2634

2735
- Declare which AI models your application needs
2836
- Specify model configurations and requirements
@@ -66,7 +74,14 @@ models:
6674
Common configuration options include:
6775
- `model` (required): The OCI artifact identifier for the model. This is what Compose pulls and runs via the model runner.
6876
- `context_size`: Defines the maximum token context size for the model.
77+
78+
> [!NOTE]
79+
> Each model has its own maximum context size. When increasing the context length,
80+
> consider your hardware constraints. In general, try to use the smallest context size
81+
> possible for your use case.
82+
6983
- `runtime_flags`: A list of raw command-line flags passed to the inference engine when the model is started.
84+
For example, if If you use llama.cpp, you can pass any of [the available parameters](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md).
7085
- Platform-specific options may also be available via extensions attributes `x-*`
7186

7287
## Service model binding
@@ -131,25 +146,58 @@ One of the key benefits of using Compose models is portability across different
131146

132147
### Docker Model Runner
133148

134-
When Docker Model Runner is enabled:
149+
When [Docker Model Runner is enabled](/manuals/ai/model-runner/_index.md):
135150

136151
```yaml
137152
services:
138153
chat-app:
139154
image: my-chat-app
140155
models:
141-
- llm
156+
llm:
157+
endpoint_var: AI_MODEL_URL
158+
model_var: AI_MODEL_NAME
142159
143160
models:
144161
llm:
145162
model: ai/smollm2
163+
context_size: 4096
164+
runtime_flags:
165+
- "--no-prefill-assistant"
146166
```
147167

148168
Docker Model Runner will:
149169
- Pull and run the specified model locally
150170
- Provide endpoint URLs for accessing the model
151171
- Inject environment variables into the service
152172

173+
#### Alternative configuration with Provider services
174+
175+
> [!TIP]
176+
>
177+
> This approach is deprecated. Use the [`models` top-level element](#use-models-definition) instead.
178+
179+
You can also use the `provider` service type, which allows you to declare platform capabilities required by your application.
180+
For AI models, you can use the `model` type to declare model dependencies.
181+
182+
To define a model provider:
183+
184+
```yaml
185+
services:
186+
chat:
187+
image: my-chat-app
188+
depends_on:
189+
- ai_runner
190+
191+
ai_runner:
192+
provider:
193+
type: model
194+
options:
195+
model: ai/smollm2
196+
context-size: 1024
197+
runtime-flags: "--no-prefill-assistant"
198+
```
199+
200+
153201
### Cloud providers
154202

155203
The same Compose file can run on cloud providers that support Compose models:
@@ -181,4 +229,4 @@ Cloud providers might:
181229
- [`models` top-level element](/reference/compose-file/models.md)
182230
- [`models` attribute](/reference/compose-file/services.md#models)
183231
- [Docker Model Runner documentation](/manuals/ai/model-runner.md)
184-
- [Compose Model Runner documentation](/manuals/ai/compose/model-runner.md)
232+
- [Compose Model Runner documentation](/manuals/ai/compose/models-and-compose.md)

content/manuals/ai/model-runner/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ Models are pulled from Docker Hub the first time they're used and stored locally
8484
> Using Testcontainers or Docker Compose?
8585
> [Testcontainers for Java](https://java.testcontainers.org/modules/docker_model_runner/)
8686
> and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/), and
87-
> [Docker Compose](/manuals/ai/compose/model-runner.md) now support Docker Model Runner.
87+
> [Docker Compose](/manuals/ai/compose/models-and-compose.md) now support Docker Model Runner.
8888
8989
## Enable Docker Model Runner
9090

content/manuals/desktop/release-notes.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -219,7 +219,7 @@ For more frequently asked questions, see the [FAQs](/manuals/desktop/troubleshoo
219219
- Docker Model Runner is now available on x86 Windows machines with NVIDIA GPUs.
220220
- You can now [push models](/manuals/ai/model-runner.md#push-a-model-to-docker-hub) to Docker Hub with Docker Model Runner.
221221
- Added support for Docker Model Runner's model management and chat interface in Docker Desktop for Mac and Windows (on hardware supporting Docker Model Runner). Users can now view, interact with, and manage local AI models through a new dedicated interface.
222-
- [Docker Compose](/manuals/ai/compose/model-runner.md) and Testcontainers [Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner.
222+
- [Docker Compose](/manuals/ai/compose/models-and-compose.md) and Testcontainers [Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner.
223223
- Introducing Docker Desktop in the [Microsoft App Store](https://apps.microsoft.com/detail/xp8cbj40xlbwkx?hl=en-GB&gl=GB).
224224

225225
### Upgrades

0 commit comments

Comments
 (0)