Skip to content

Commit 2c12a52

Browse files
ArthurFlagcrazy-max
authored andcommitted
remove manual docs
Signed-off-by: CrazyMax <[email protected]>
1 parent e9783a2 commit 2c12a52

File tree

1 file changed

+4
-185
lines changed

1 file changed

+4
-185
lines changed

content/manuals/ai/model-runner.md

Lines changed: 4 additions & 185 deletions
Original file line numberDiff line numberDiff line change
@@ -53,191 +53,6 @@ You can now use the `docker model` command in the CLI and view and interact with
5353
apt install docker-model-plugin
5454
```
5555

56-
## Available commands
57-
58-
### Model runner status
59-
60-
Check whether the Docker Model Runner is active and displays the current inference engine:
61-
62-
```console
63-
$ docker model status
64-
```
65-
66-
### View all commands
67-
68-
Displays help information and a list of available subcommands.
69-
70-
```console
71-
$ docker model help
72-
```
73-
74-
Output:
75-
76-
```text
77-
Usage: docker model COMMAND
78-
79-
Commands:
80-
list List models available locally
81-
pull Download a model from Docker Hub
82-
rm Remove a downloaded model
83-
run Run a model interactively or with a prompt
84-
status Check if the model runner is running
85-
version Show the current version
86-
```
87-
88-
### Pull a model
89-
90-
Pulls a model from Docker Hub to your local environment.
91-
92-
```console
93-
$ docker model pull <model>
94-
```
95-
96-
Example:
97-
98-
```console
99-
$ docker model pull ai/smollm2
100-
```
101-
102-
Output:
103-
104-
```text
105-
Downloaded: 257.71 MB
106-
Model ai/smollm2 pulled successfully
107-
```
108-
109-
The models also display in the Docker Desktop Dashboard.
110-
111-
#### Pull from Hugging Face
112-
113-
You can also pull GGUF models directly from [Hugging Face](https://huggingface.co/models?library=gguf).
114-
115-
```console
116-
$ docker model pull hf.co/<model-you-want-to-pull>
117-
```
118-
119-
For example:
120-
121-
```console
122-
$ docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF
123-
```
124-
125-
Pulls the [bartowski/Llama-3.2-1B-Instruct-GGUF](https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF).
126-
127-
### List available models
128-
129-
Lists all models currently pulled to your local environment.
130-
131-
```console
132-
$ docker model list
133-
```
134-
135-
You will see something similar to:
136-
137-
```text
138-
+MODEL PARAMETERS QUANTIZATION ARCHITECTURE MODEL ID CREATED SIZE
139-
+ai/smollm2 361.82 M IQ2_XXS/Q4_K_M llama 354bf30d0aa3 3 days ago 256.35 MiB
140-
```
141-
142-
### Run a model
143-
144-
Run a model and interact with it using a submitted prompt or in chat mode. When you run a model, Docker
145-
calls an Inference Server API endpoint hosted by the Model Runner through Docker Desktop. The model
146-
stays in memory until another model is requested, or until a pre-defined inactivity timeout is reached (currently 5 minutes).
147-
148-
You do not have to use `Docker model run` before interacting with a specific model from a
149-
host process or from within a container. Model Runner transparently loads the requested model on-demand, assuming it has been
150-
pulled beforehand and is locally available.
151-
152-
#### One-time prompt
153-
154-
```console
155-
$ docker model run ai/smollm2 "Hi"
156-
```
157-
158-
Output:
159-
160-
```text
161-
Hello! How can I assist you today?
162-
```
163-
164-
#### Interactive chat
165-
166-
```console
167-
$ docker model run ai/smollm2
168-
```
169-
170-
Output:
171-
172-
```text
173-
Interactive chat mode started. Type '/bye' to exit.
174-
> Hi
175-
Hi there! It's SmolLM, AI assistant. How can I help you today?
176-
> /bye
177-
Chat session ended.
178-
```
179-
180-
> [!TIP]
181-
>
182-
> You can also use chat mode in the Docker Desktop Dashboard when you select the model in the **Models** tab.
183-
184-
### Push a model to Docker Hub
185-
186-
To push your model to Docker Hub:
187-
188-
```console
189-
$ docker model push <namespace>/<model>
190-
```
191-
192-
### Tag a model
193-
194-
To specify a particular version or variant of the model:
195-
196-
```console
197-
$ docker model tag
198-
```
199-
200-
If no tag is provided, Docker defaults to `latest`.
201-
202-
### View the logs
203-
204-
Fetch logs from Docker Model Runner to monitor activity or debug issues.
205-
206-
```console
207-
$ docker model logs
208-
```
209-
210-
The following flags are accepted:
211-
212-
- `-f`/`--follow`: View logs with real-time streaming
213-
- `--no-engines`: Exclude inference engine logs from the output
214-
215-
### Remove a model
216-
217-
Removes a downloaded model from your system.
218-
219-
```console
220-
$ docker model rm <model>
221-
```
222-
223-
Output:
224-
225-
```text
226-
Model <model> removed successfully
227-
```
228-
229-
### Package a model
230-
231-
Packages a GGUF file into a Docker model OCI artifact, with optional licenses, and pushes it to the specified registry.
232-
233-
```console
234-
$ docker model package \
235-
--gguf ./model.gguf \
236-
--licenses license1.txt \
237-
--licenses license2.txt \
238-
--push registry.example.com/ai/custom-model
239-
```
240-
24156
## Integrate the Docker Model Runner into your software development lifecycle
24257

24358
You can now start building your Generative AI application powered by the Docker Model Runner.
@@ -266,6 +81,10 @@ You can now interact with your own GenAI app, powered by a local model. Try a fe
26681

26782
All the available models are hosted in the [public Docker Hub namespace of `ai`](https://hub.docker.com/u/ai).
26883

84+
### What CLI commands are available?
85+
86+
See [the reference docs](/reference/cli/docker/model/).
87+
26988
### What API endpoints are available?
27089

27190
Once the feature is enabled, new API endpoints are available under the following base URLs:

0 commit comments

Comments
 (0)