Skip to content

Commit 5afc72f

Browse files
authored
dmr: request inspector (#23216)
<!--Delete sections as needed --> ## Description <!-- Tell us what you did and why --> ## Related issues or tickets <!-- Related issues, pull requests, or Jira tickets --> ## Reviews <!-- Notes for reviewers here --> <!-- List applicable reviews (optionally @tag reviewers) --> - [ ] Technical review - [ ] Editorial review - [ ] Product review
1 parent de37c04 commit 5afc72f

File tree

1 file changed

+24
-0
lines changed

1 file changed

+24
-0
lines changed

content/manuals/ai/model-runner/_index.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ with AI models locally.
3939
- Package GGUF files as OCI Artifacts and publish them to any Container Registry
4040
- Run and interact with AI models directly from the command line or from the Docker Desktop GUI
4141
- Manage local models and display logs
42+
- Display prompts and responses details
4243

4344
## Requirements
4445

@@ -217,6 +218,7 @@ Use the [`docker model run` command](/reference/cli/docker/model/run/).
217218

218219
## Troubleshooting
219220

221+
### Display the logs
220222
To troubleshoot issues, display the logs:
221223

222224
{{< tabs group="release" >}}
@@ -234,6 +236,28 @@ Use the [`docker model logs` command](/reference/cli/docker/model/logs/).
234236
{{< /tab >}}
235237
{{< /tabs >}}
236238

239+
### Inpect requests and responses
240+
241+
Inspecting requests and responses helps you diagnose model-related issues.
242+
For example, you can evaluate context usage to verify you stay within the model's context
243+
window or display the full body of a request to control the parameters you are passing to your models
244+
when developing with a framework.
245+
246+
In Docker Desktop, to inspect the requests and responses for each model:
247+
248+
1. Select **Models** and select the **Requests** tab. This view displays all the requests to all models:
249+
- The time the request was sent.
250+
- The model name and version
251+
- The prompt/request
252+
- The context usage
253+
- The time it took for the response to be generated.
254+
2. Select one of the requests to display further details:
255+
- In the **Overview** tab, view the token usage, response metadata and generation speed, and the actual prompt and response.
256+
- In the **Request** and **Response** tabs, view the full JSON payload of the request and the response.
257+
258+
> [!NOTE]
259+
> You can also display the requests for a specific model when you select a model and then select the **Requests** tab.
260+
237261
## Publish a model
238262

239263
> [!NOTE]

0 commit comments

Comments
 (0)