Skip to content

Commit 2cc416f

Browse files
committed
dmr: request inspector
1 parent 7fbe6e1 commit 2cc416f

File tree

1 file changed

+26
-1
lines changed

1 file changed

+26
-1
lines changed

content/manuals/ai/model-runner/_index.md

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ with AI models locally.
3939
- Package GGUF files as OCI Artifacts and publish them to any Container Registry
4040
- Run and interact with AI models directly from the command line or from the Docker Desktop GUI
4141
- Manage local models and display logs
42+
- Display prompts and responses details
4243

4344
## Requirements
4445

@@ -188,7 +189,7 @@ docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF
188189
{{< tabs group="release" >}}
189190
{{< tab name="From Docker Desktop">}}
190191

191-
1. Select **Models** and select the **Local** tab
192+
1. Select **Models** and select the **Local** tab.
192193
1. Click the play button. The interactive chat screen opens.
193194

194195
![screencapture of the Local view](./images/dmr-run.png)
@@ -203,6 +204,8 @@ Use the [`docker model run` command](/reference/cli/docker/model/run/).
203204

204205
## Troubleshooting
205206

207+
### Display the logs
208+
206209
To troubleshoot potential issues, display the logs:
207210

208211
{{< tabs group="release" >}}
@@ -220,6 +223,28 @@ Use the [`docker model logs` command](/reference/cli/docker/model/logs/).
220223
{{< /tab >}}
221224
{{< /tabs >}}
222225

226+
### Inpect requests and responses
227+
228+
Inspecting requests and responses helps you diagnose model-related issues.
229+
For example, you can evaluate context usage to verify you stay within the model's context
230+
window or display the full body of a request to control the parameters you are passing to your models
231+
when developing with a framework.
232+
233+
In Docker Desktop, to inspect the requests and responses for each model:
234+
235+
1. Select **Models** and select the **Requests** tab. This view displays all the requests to all models:
236+
- The time the request was sent.
237+
- The model name and version
238+
- The prompt/request
239+
- The context usage
240+
- The time it took for the response to be generated.
241+
2. Select one of the requests to display further details:
242+
- In the **Overview** tab, view the token usage, response metadata and generation speed, and the actual prompt and response.
243+
- In the **Request** and **Response** tabs, view the full JSON payload of the request and the response.
244+
245+
> [!NOTE]
246+
> You can also display the requests for a specific model when you select a model and then select the **Requests** tab.
247+
223248
## Publish a model
224249

225250
> [!NOTE]

0 commit comments

Comments
 (0)