Skip to content

Commit ed76536

Browse files
committed
remove manual docs
1 parent 349fbb4 commit ed76536

File tree

1 file changed

+4
-185
lines changed

1 file changed

+4
-185
lines changed

content/manuals/ai/model-runner.md

Lines changed: 4 additions & 185 deletions
Original file line numberDiff line numberDiff line change
@@ -42,191 +42,6 @@ Models are pulled from Docker Hub the first time they're used and stored locally
4242

4343
You can now use the `docker model` command in the CLI and view and interact with your local models in the **Models** tab in the Docker Desktop Dashboard.
4444

45-
## Available commands
46-
47-
### Model runner status
48-
49-
Check whether the Docker Model Runner is active and displays the current inference engine:
50-
51-
```console
52-
$ docker model status
53-
```
54-
55-
### View all commands
56-
57-
Displays help information and a list of available subcommands.
58-
59-
```console
60-
$ docker model help
61-
```
62-
63-
Output:
64-
65-
```text
66-
Usage: docker model COMMAND
67-
68-
Commands:
69-
list List models available locally
70-
pull Download a model from Docker Hub
71-
rm Remove a downloaded model
72-
run Run a model interactively or with a prompt
73-
status Check if the model runner is running
74-
version Show the current version
75-
```
76-
77-
### Pull a model
78-
79-
Pulls a model from Docker Hub to your local environment.
80-
81-
```console
82-
$ docker model pull <model>
83-
```
84-
85-
Example:
86-
87-
```console
88-
$ docker model pull ai/smollm2
89-
```
90-
91-
Output:
92-
93-
```text
94-
Downloaded: 257.71 MB
95-
Model ai/smollm2 pulled successfully
96-
```
97-
98-
The models also display in the Docker Desktop Dashboard.
99-
100-
#### Pull from Hugging Face
101-
102-
You can also pull GGUF models directly from [Hugging Face](https://huggingface.co/models?library=gguf).
103-
104-
```console
105-
$ docker model pull hf.co/<model-you-want-to-pull>
106-
```
107-
108-
For example:
109-
110-
```console
111-
$ docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF
112-
```
113-
114-
Pulls the [bartowski/Llama-3.2-1B-Instruct-GGUF](https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF).
115-
116-
### List available models
117-
118-
Lists all models currently pulled to your local environment.
119-
120-
```console
121-
$ docker model list
122-
```
123-
124-
You will see something similar to:
125-
126-
```text
127-
+MODEL PARAMETERS QUANTIZATION ARCHITECTURE MODEL ID CREATED SIZE
128-
+ai/smollm2 361.82 M IQ2_XXS/Q4_K_M llama 354bf30d0aa3 3 days ago 256.35 MiB
129-
```
130-
131-
### Run a model
132-
133-
Run a model and interact with it using a submitted prompt or in chat mode. When you run a model, Docker
134-
calls an Inference Server API endpoint hosted by the Model Runner through Docker Desktop. The model
135-
stays in memory until another model is requested, or until a pre-defined inactivity timeout is reached (currently 5 minutes).
136-
137-
You do not have to use `Docker model run` before interacting with a specific model from a
138-
host process or from within a container. Model Runner transparently loads the requested model on-demand, assuming it has been
139-
pulled beforehand and is locally available.
140-
141-
#### One-time prompt
142-
143-
```console
144-
$ docker model run ai/smollm2 "Hi"
145-
```
146-
147-
Output:
148-
149-
```text
150-
Hello! How can I assist you today?
151-
```
152-
153-
#### Interactive chat
154-
155-
```console
156-
$ docker model run ai/smollm2
157-
```
158-
159-
Output:
160-
161-
```text
162-
Interactive chat mode started. Type '/bye' to exit.
163-
> Hi
164-
Hi there! It's SmolLM, AI assistant. How can I help you today?
165-
> /bye
166-
Chat session ended.
167-
```
168-
169-
> [!TIP]
170-
>
171-
> You can also use chat mode in the Docker Desktop Dashboard when you select the model in the **Models** tab.
172-
173-
### Push a model to Docker Hub
174-
175-
To push your model to Docker Hub:
176-
177-
```console
178-
$ docker model push <namespace>/<model>
179-
```
180-
181-
### Tag a model
182-
183-
To specify a particular version or variant of the model:
184-
185-
```console
186-
$ docker model tag
187-
```
188-
189-
If no tag is provided, Docker defaults to `latest`.
190-
191-
### View the logs
192-
193-
Fetch logs from Docker Model Runner to monitor activity or debug issues.
194-
195-
```console
196-
$ docker model logs
197-
```
198-
199-
The following flags are accepted:
200-
201-
- `-f`/`--follow`: View logs with real-time streaming
202-
- `--no-engines`: Exclude inference engine logs from the output
203-
204-
### Remove a model
205-
206-
Removes a downloaded model from your system.
207-
208-
```console
209-
$ docker model rm <model>
210-
```
211-
212-
Output:
213-
214-
```text
215-
Model <model> removed successfully
216-
```
217-
218-
### Package a model
219-
220-
Packages a GGUF file into a Docker model OCI artifact, with optional licenses, and pushes it to the specified registry.
221-
222-
```console
223-
$ docker model package \
224-
--gguf ./model.gguf \
225-
--licenses license1.txt \
226-
--licenses license2.txt \
227-
--push registry.example.com/ai/custom-model
228-
```
229-
23045
## Integrate the Docker Model Runner into your software development lifecycle
23146

23247
You can now start building your Generative AI application powered by the Docker Model Runner.
@@ -255,6 +70,10 @@ You can now interact with your own GenAI app, powered by a local model. Try a fe
25570

25671
All the available models are hosted in the [public Docker Hub namespace of `ai`](https://hub.docker.com/u/ai).
25772

73+
### What CLI commands are available?
74+
75+
See [the reference docs](/reference/cli/docker/model/).
76+
25877
### What API endpoints are available?
25978

26079
Once the feature is enabled, new API endpoints are available under the following base URLs:

0 commit comments

Comments
 (0)