@@ -53,191 +53,6 @@ You can now use the `docker model` command in the CLI and view and interact with
5353 apt install docker-model-plugin
5454 ```
5555
56- ## Available commands
57-
58- ### Model runner status
59-
60- Check whether the Docker Model Runner is active and displays the current inference engine:
61-
62- ``` console
63- $ docker model status
64- ```
65-
66- ### View all commands
67-
68- Displays help information and a list of available subcommands.
69-
70- ``` console
71- $ docker model help
72- ```
73-
74- Output:
75-
76- ``` text
77- Usage: docker model COMMAND
78-
79- Commands:
80- list List models available locally
81- pull Download a model from Docker Hub
82- rm Remove a downloaded model
83- run Run a model interactively or with a prompt
84- status Check if the model runner is running
85- version Show the current version
86- ```
87-
88- ### Pull a model
89-
90- Pulls a model from Docker Hub to your local environment.
91-
92- ``` console
93- $ docker model pull < model>
94- ```
95-
96- Example:
97-
98- ``` console
99- $ docker model pull ai/smollm2
100- ```
101-
102- Output:
103-
104- ``` text
105- Downloaded: 257.71 MB
106- Model ai/smollm2 pulled successfully
107- ```
108-
109- The models also display in the Docker Desktop Dashboard.
110-
111- #### Pull from Hugging Face
112-
113- You can also pull GGUF models directly from [ Hugging Face] ( https://huggingface.co/models?library=gguf ) .
114-
115- ``` console
116- $ docker model pull hf.co/< model-you-want-to-pull>
117- ```
118-
119- For example:
120-
121- ``` console
122- $ docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF
123- ```
124-
125- Pulls the [ bartowski/Llama-3.2-1B-Instruct-GGUF] ( https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF ) .
126-
127- ### List available models
128-
129- Lists all models currently pulled to your local environment.
130-
131- ``` console
132- $ docker model list
133- ```
134-
135- You will see something similar to:
136-
137- ``` text
138- +MODEL PARAMETERS QUANTIZATION ARCHITECTURE MODEL ID CREATED SIZE
139- +ai/smollm2 361.82 M IQ2_XXS/Q4_K_M llama 354bf30d0aa3 3 days ago 256.35 MiB
140- ```
141-
142- ### Run a model
143-
144- Run a model and interact with it using a submitted prompt or in chat mode. When you run a model, Docker
145- calls an Inference Server API endpoint hosted by the Model Runner through Docker Desktop. The model
146- stays in memory until another model is requested, or until a pre-defined inactivity timeout is reached (currently 5 minutes).
147-
148- You do not have to use ` Docker model run ` before interacting with a specific model from a
149- host process or from within a container. Model Runner transparently loads the requested model on-demand, assuming it has been
150- pulled beforehand and is locally available.
151-
152- #### One-time prompt
153-
154- ``` console
155- $ docker model run ai/smollm2 " Hi"
156- ```
157-
158- Output:
159-
160- ``` text
161- Hello! How can I assist you today?
162- ```
163-
164- #### Interactive chat
165-
166- ``` console
167- $ docker model run ai/smollm2
168- ```
169-
170- Output:
171-
172- ``` text
173- Interactive chat mode started. Type '/bye' to exit.
174- > Hi
175- Hi there! It's SmolLM, AI assistant. How can I help you today?
176- > /bye
177- Chat session ended.
178- ```
179-
180- > [ !TIP]
181- >
182- > You can also use chat mode in the Docker Desktop Dashboard when you select the model in the ** Models** tab.
183-
184- ### Push a model to Docker Hub
185-
186- To push your model to Docker Hub:
187-
188- ``` console
189- $ docker model push < namespace> /< model>
190- ```
191-
192- ### Tag a model
193-
194- To specify a particular version or variant of the model:
195-
196- ``` console
197- $ docker model tag
198- ```
199-
200- If no tag is provided, Docker defaults to ` latest ` .
201-
202- ### View the logs
203-
204- Fetch logs from Docker Model Runner to monitor activity or debug issues.
205-
206- ``` console
207- $ docker model logs
208- ```
209-
210- The following flags are accepted:
211-
212- - ` -f ` /` --follow ` : View logs with real-time streaming
213- - ` --no-engines ` : Exclude inference engine logs from the output
214-
215- ### Remove a model
216-
217- Removes a downloaded model from your system.
218-
219- ``` console
220- $ docker model rm < model>
221- ```
222-
223- Output:
224-
225- ``` text
226- Model <model> removed successfully
227- ```
228-
229- ### Package a model
230-
231- Packages a GGUF file into a Docker model OCI artifact, with optional licenses, and pushes it to the specified registry.
232-
233- ``` console
234- $ docker model package \
235- --gguf ./model.gguf \
236- --licenses license1.txt \
237- --licenses license2.txt \
238- --push registry.example.com/ai/custom-model
239- ```
240-
24156## Integrate the Docker Model Runner into your software development lifecycle
24257
24358You can now start building your Generative AI application powered by the Docker Model Runner.
@@ -266,6 +81,10 @@ You can now interact with your own GenAI app, powered by a local model. Try a fe
26681
26782All the available models are hosted in the [ public Docker Hub namespace of ` ai ` ] ( https://hub.docker.com/u/ai ) .
26883
84+ ### What CLI commands are available?
85+
86+ See [ the reference docs] ( /reference/cli/docker/model/ ) .
87+
26988### What API endpoints are available?
27089
27190Once the feature is enabled, new API endpoints are available under the following base URLs:
0 commit comments