diff --git a/docs/product/insights/ai/agents/dashboard.mdx b/docs/product/insights/ai/agents/dashboard.mdx index e9179f1a2fc72..2d3e27e7e167a 100644 --- a/docs/product/insights/ai/agents/dashboard.mdx +++ b/docs/product/insights/ai/agents/dashboard.mdx @@ -5,33 +5,50 @@ description: Learn how to use Sentry's AI Agents Dashboard. og_image: /og-images/product-insights-ai-agents-dashboard.png --- -Once you've [configured the Sentry SDK](/product/insights/ai/agents/getting-started/) for your AI agent project, you'll start receiving data in the Sentry [AI Agents Insights](https://sentry.io/orgredirect/organizations/:orgslug/insights/ai/agents/) dashboard. +The [AI Agents Insights](https://sentry.io/orgredirect/organizations/:orgslug/insights/ai/agents/) dashboard provides a comprehensive view of your AI workflows, including executions, model costs and token usage, tool calls, and recent errors. After you've [configured the Sentry SDK](/product/insights/ai/agents/getting-started/) in your project, telemetry data is collected and displayed in the dashboard to support analysis of system behavior and performance. -The main dashboard provides a comprehensive view of all your AI agent activities, performance metrics, and recent executions. +At the top of the page, you will be able to navigate between the three tabs: [Overview](#overview), [Models](#models) and [Tools](#tools). -![AI Agents Monitoring Overview](./img/overview.png) +## Overview -The dashboard displays key widgets like: +The Overview tab is the landing page for monitoring your AI workflows: + +![AI Agents Monitoring Overview](./img/overview-tab.png) + +The dashboard displays the following key widgets: - **Traffic**: Shows agent runs over time, error rates, and releases to track overall activity and health - **Duration**: Displays response times for your agent executions to monitor performance - **Issues**: Highlights recent errors and problems that need attention, including agent failures and exceptions +- **LLM Calls**: Count of LLM generations over time +- **Tokens Used**: Token usage by top models +- **Tool Calls**: Tool call volume and trends + +Below these widgets is a traces table with detailed distribution information: + +![AI Agent Trace Table](./img/trace-table-detailed-distribution.png) + +Click on any trace to open the [abbreviated trace view](#abbreviated-trace-view) in a drawer. + +## Models + +The Models tab displays Model Cost, Tokens Used, and Token Types widgets, as well as all used models with durations and token usage: + +![AI Agents Monitoring Models](./img/models-tab.png) + -Underneath these widgets are three segments that offer focused widgets and a detailed distribution table: +## Tools -- **Traces**: LLM Calls, Tokens Used, and Tool Calls widgets, as well as recent agent runs with duration, errors, number of LLM and tool calls, and token usage -- **Models**: Model Cost, Tokens Used, and Token Types widgets, as well as all used models with durations and token usage -- **Tools**: Tool Calls and Tool Errors widgets, as well as all used tools with durations and errors +The Tools tab displays Tool Calls and Tool Errors widgets, as well as all used tools with durations and errors: -![AI Agent Trace Table](./img/trace-table.png) +![AI Agents Monitoring Tools](./img/tools-tab.png) -Click on any trace to open the abbreviated trace view in a drawer. ## Abbreviated Trace View Opens as a drawer when clicking any trace, showing essential details: -![AI Agent Abbreviated Trace View](./img/abbreviated-trace-view.png) +![AI Agent Abbreviated Trace View](./img/abbreviated-trace-view-drawer.png) - **Agent Invocations**: Each agent execution and nested calls - **LLM Generations**: Language model interactions with token breakdown @@ -46,7 +63,7 @@ Click **"View in full trace"** for comprehensive debugging details. Shows complete agent workflow with full context: -![AI Agent Detailed Trace View](./img/trace-view.png) +![AI Agent Detailed Trace View](./img/full-trace-view.png) This detailed view reveals: diff --git a/docs/product/insights/ai/agents/img/abbreviated-trace-view-drawer.png b/docs/product/insights/ai/agents/img/abbreviated-trace-view-drawer.png new file mode 100644 index 0000000000000..1552511e58efd Binary files /dev/null and b/docs/product/insights/ai/agents/img/abbreviated-trace-view-drawer.png differ diff --git a/docs/product/insights/ai/agents/img/abbreviated-trace-view.png b/docs/product/insights/ai/agents/img/abbreviated-trace-view.png deleted file mode 100644 index 0dd4332d1fb0f..0000000000000 Binary files a/docs/product/insights/ai/agents/img/abbreviated-trace-view.png and /dev/null differ diff --git a/docs/product/insights/ai/agents/img/full-trace-view.png b/docs/product/insights/ai/agents/img/full-trace-view.png new file mode 100644 index 0000000000000..f233c39b6a182 Binary files /dev/null and b/docs/product/insights/ai/agents/img/full-trace-view.png differ diff --git a/docs/product/insights/ai/agents/img/models-tab.png b/docs/product/insights/ai/agents/img/models-tab.png new file mode 100644 index 0000000000000..53b1aeb770ee6 Binary files /dev/null and b/docs/product/insights/ai/agents/img/models-tab.png differ diff --git a/docs/product/insights/ai/agents/img/overview-tab.png b/docs/product/insights/ai/agents/img/overview-tab.png new file mode 100644 index 0000000000000..5b46828d4c0f8 Binary files /dev/null and b/docs/product/insights/ai/agents/img/overview-tab.png differ diff --git a/docs/product/insights/ai/agents/img/overview.png b/docs/product/insights/ai/agents/img/overview.png deleted file mode 100644 index 4985f2b3e1f7b..0000000000000 Binary files a/docs/product/insights/ai/agents/img/overview.png and /dev/null differ diff --git a/docs/product/insights/ai/agents/img/tools-tab.png b/docs/product/insights/ai/agents/img/tools-tab.png new file mode 100644 index 0000000000000..6335cbed8d57f Binary files /dev/null and b/docs/product/insights/ai/agents/img/tools-tab.png differ diff --git a/docs/product/insights/ai/agents/img/trace-table-detailed-distribution.png b/docs/product/insights/ai/agents/img/trace-table-detailed-distribution.png new file mode 100644 index 0000000000000..03e5a618edac7 Binary files /dev/null and b/docs/product/insights/ai/agents/img/trace-table-detailed-distribution.png differ diff --git a/docs/product/insights/ai/agents/img/trace-table.png b/docs/product/insights/ai/agents/img/trace-table.png deleted file mode 100644 index 4f35744a88194..0000000000000 Binary files a/docs/product/insights/ai/agents/img/trace-table.png and /dev/null differ diff --git a/docs/product/insights/ai/agents/img/trace-view.png b/docs/product/insights/ai/agents/img/trace-view.png deleted file mode 100644 index 381e2b1f4a163..0000000000000 Binary files a/docs/product/insights/ai/agents/img/trace-view.png and /dev/null differ diff --git a/docs/product/insights/ai/agents/index.mdx b/docs/product/insights/ai/agents/index.mdx index 3d8aac8def691..ae0437c2e437e 100644 --- a/docs/product/insights/ai/agents/index.mdx +++ b/docs/product/insights/ai/agents/index.mdx @@ -19,6 +19,6 @@ Sentry's AI Agent monitoring tools help you understand what's going on with your To use AI Agent Monitoring, you must have an existing Sentry account and project set up. If you don't have one, [create an account here](https://sentry.io/signup/). -![AI Agents Monitoring Overview](./img/overview.png) +![AI Agents Monitoring Overview](./img/overview-tab.png) Learn how to [set up Sentry for AI Agents](/product/insights/ai/agents/getting-started/).