Skip to content

Commit a358a36

Browse files
antonpirkerantonpirker
committed
work
1 parent 1f69116 commit a358a36

File tree

1 file changed

+97
-34
lines changed
  • docs/platforms/python/tracing/instrumentation/custom-instrumentation

1 file changed

+97
-34
lines changed

‎docs/platforms/python/tracing/instrumentation/custom-instrumentation/agents-module.mdx‎

Lines changed: 97 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,87 @@ sidebar_order: 500
44
description: "Learn how to manually instrument your code to use Sentry's Agents module."
55
---
66

7-
As a prerequisite to setting up [AI Agents](/product/insights/agents/), you’ll need to first <PlatformLink to="/tracing/">set up tracing</PlatformLink>. Once this is done, the Python SDK will automatically instrument AI agents created with the `openai-agents` library. If that doesn't fit your use case, you can set up using custom instrumentation described below.
7+
As a prerequisite to setting up [AI Agents](https://sentry.io/orgredirect/organizations/:orgslug/insights/agents/), you’ll need to first <PlatformLink to="/tracing/">set up tracing</PlatformLink>. Once this is done, the Python SDK will automatically instrument AI agents created with the `openai-agents` library. If that doesn't fit your use case, you can set up using custom instrumentation described below.
88

99
## Custom Instrumentation
1010

11-
For your AI agents data to show up in the Sentry Agents Insights Module a couple of different spans can be created. Those spans need to have well defined names and attributes.
11+
For your AI agents data to show up in the Sentry Agents Insights Module a couple of different spans must be created. Those spans need to have well defined names and attributes.
1212

13-
### Common Span Attributes
13+
## Span Types
14+
15+
### Invoke Agent Span
16+
17+
Describes AI agent invocation.
18+
19+
- The spans `op` MUST be `"gen_ai.invoke_agent"`.
20+
- The span `name` MUST be `"invoke_agent {gen_ai.agent.name}"`.
21+
- The `gen_ai.operation.name` attribute MUST be `"invoke_agent"`.
22+
- The `gen_ai.agent.name` attribute SHOULD be set to the agents name. (e.g. `"Weather Agent"`)
23+
24+
### Execute Tool Span
25+
26+
Describes tool execution span.
27+
28+
- The spans `op` MUST be `"gen_ai.execute_tool"`.
29+
- The spans `name` MUST be `"gen_ai.execute_tool {gen_ai.tool.name}"`. (e.g. `"gen_ai.execute_tool query_database"`)
30+
- The `gen_ai.tool.name` attribute SHOULD be set to the name of the tool. (e.g. `"query_database"`)
31+
32+
Additional attributes on the span:
33+
34+
| Data Attribute | Type | Requirement Level | Description | Example |
35+
| :------------------------ | :----- | :---------------- | :--------------------------------------------------- | :----------------------------------------- |
36+
| `gen_ai.tool.description` | string | optional | Description of the tool executed. | `"Tool returning a random number"` |
37+
| `gen_ai.tool.input` | string | optional | Input that was given to the executed tool as string. | `"{\"max\":10}"` |
38+
| `gen_ai.tool.name:` | string | optional | Name of the tool executed. | `"random_number"` |
39+
| `gen_ai.tool.output` | string | optional | The output from the tool. | `"7"` |
40+
| `gen_ai.tool.type` | string | optional | The type of the tools. | `"function"`; `"extension"`; `"datastore"` |
41+
42+
### Handoff Span
43+
44+
A span that describes the handoff from one agent to another agent.
45+
46+
- The spans `op` MUST be `"gen_ai.handoff"`.
47+
- The spans `name` SHOULD be `"handoff from {from_agent} to {from_agent}"`.
48+
49+
### AI Client Span
50+
51+
This span represents a client call to an AI model or service that generates a response or requests a tool call based on the input prompt.
52+
53+
- The span `op` MUST be `"gen_ai.{gen_ai.operation.name}"`. (e.g. `"gen_ai.chat"`)
54+
- The span `name` MUST be `{gen_ai.operation.name} {gen_ai.request.model}"`. (e.g. `"chat o3-mini"`)
55+
56+
Additional attributes on the span:
57+
58+
| Data Attribute | Type | Requirement Level | Description | Example |
59+
| :------------------------------------- | :----- | :---------------- | :--------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------- |
60+
| `gen_ai.request.available_tools` | string | optional | List of objects describing the available tools. **[0]** | `"[{\"name\": \"random_number\", \"description\": \"...\"}, {\"name\": \"query_db\", \"description\": \"...\"}]"` |
61+
| `gen_ai.request.frequency_penalty` | float | optional | Model configuration parameter. | `0.5` |
62+
| `gen_ai.request.max_tokens` | int | optional | Model configuration parameter. | `500` |
63+
| `gen_ai.request.messages` | string | optional | List of objects describing the messages (prompts) sent to the LLM **[0]**, **[1]** | `"[{\"role\": \"system\", \"content\": [{...}]}, {\"role\": \"system\", \"content\": [{...}]}]"` |
64+
| `gen_ai.request.presence_penalty` | float | optional | Model configuration parameter. | `0.5` |
65+
| `gen_ai.request.temperature` | float | optional | Model configuration parameter. | `0.1` |
66+
| `gen_ai.request.top_p` | float | optional | Model configuration parameter. | `0.7` |
67+
| `gen_ai.response.tool_calls` | string | optional | The tool calls in the model’s response. **[0]** | `"[{\"name\": \"random_number\", \"type\": \"function_call\", \"arguments\": \"...\"}]"` |
68+
| `gen_ai.response.text` | string | optional | The text representation of the model’s responses. **[0]** | `"[\"The weather in Paris is rainy\", \"The weather in London is sunny\"]"` |
69+
| `gen_ai.usage.input_tokens.cached` | int | optional | The number of cached tokens used in the AI input (prompt) | `50` |
70+
| `gen_ai.usage.input_tokens` | int | optional | The number of tokens used in the AI input (prompt). | `10` |
71+
| `gen_ai.usage.output_tokens.reasoning` | int | optional | The number of tokens used for reasoning. | `30` |
72+
| `gen_ai.usage.output_tokens` | int | optional | The number of tokens used in the AI response. | `100` |
73+
| `gen_ai.usage.total_tokens` | int | optional | The total number of tokens used to process the prompt. (input and output) | `190` |
74+
75+
- **[0]:** As span attributes only allow primitive data types this needs to be a stringified version of a list of dictionaries. Do NOT set `[{"foo": "bar"}]` but rather the string `"[{\"foo\": \"bar\"}]"`.
76+
- **[1]:** Each item in the list has the format `{role:"", content:""}` where `role` can be `"user"`, `"assistant"`, or `"system"` and `content` can either be a string or a list of dictionaries.
77+
78+
## Common Span Attributes
1479

1580
Some attributes are common to all types of AI Agents spans:
1681

17-
| Data Attribute | Type | Description |
18-
| :---------------------- | :----- | :----------------------------------------------------------------------------------- |
19-
| `gen_ai.system` | string | The Generative AI product as identified by the client or server instrumentation. [1] |
20-
| `gen_ai.request.model` | string | The name of the AI model a request is being made to. |
21-
| `gen_ai.operation.name` | string | The name of the operation being performed. [2] |
22-
| `gen_ai.agent.name` | string | The name of the agent this span belongs to. |
82+
| Data Attribute | Type | Requirement Level | Description | Example |
83+
| :---------------------- | :----- | :---------------- | :----------------------------------------------------------------------------------- | :---------------- |
84+
| `gen_ai.system` | string | required | The Generative AI product as identified by the client or server instrumentation. [1] | `"openai"` |
85+
| `gen_ai.request.model` | string | required | The name of the AI model a request is being made to. | `"o3-mini"` |
86+
| `gen_ai.operation.name` | string | optional | The name of the operation being performed. [2] | `"chat"` |
87+
| `gen_ai.agent.name` | string | optional | The name of the agent this span belongs to. | `"Weather Agent"` |
2388

2489
**[1]** Well defined values for data attribute `gen_ai.system`:
2590

@@ -52,46 +117,44 @@ Some attributes are common to all types of AI Agents spans:
52117
| `generate_content` | Multimodal content generation operation such as Gemini Generate Content |
53118
| `invoke_agent` | Invoke GenAI agent |
54119

55-
### Invoke Agent Span
120+
### Attributes of Invoke Agent Span
56121

57122
This span wraps one invocation of an agent.
58123

59124
- `span.op` = `"gen_ai.invoke_agent"`
60125
- `span.name` = `"gen_ai.invoke_agent {gen_ai.agent.name}"` (Example: `"gen_ai.invoke_agent Weather Forecast Agent"`)
61126

62127
- Span attributes:
63-
- `gen_ai.request.model`: The model that is used.
64-
- `gen_ai.request.available_tools`: An array of objects that describe the tools available to the agent.
65-
- `gen_ai.request.frequency_penalty`: Model configuration
66-
- `gen_ai.request.max_tokens`: Model configuration
67-
- `gen_ai.request.presence_penalty`: Model configuration
68-
- `gen_ai.request.temperature`: Model configuration
69-
- `gen_ai.request.top_p`: Model configuration
70-
128+
- `gen_ai.request.model`: The model that is used.
129+
- `gen_ai.request.available_tools`: An array of objects that describe the tools available to the agent.
130+
- `gen_ai.request.frequency_penalty`: Model configuration
131+
- `gen_ai.request.max_tokens`: Model configuration
132+
- `gen_ai.request.presence_penalty`: Model configuration
133+
- `gen_ai.request.temperature`: Model configuration
134+
- `gen_ai.request.top_p`: Model configuration
71135

72-
### Execute Tool Span
136+
### Attributes of Execute Tool Span
73137

74138
This span wraps the execution of a tool.
75139

76140
- `span.op` = `"gen_ai.execute_tool"`
77141
- `span.name` = `"gen_ai.execute_tool {tool.name}"` (Example: `"gen_ai.execute_tool query_database"`)
78142

79-
80143
- Span attributes:
81-
- `gen_ai.request.available_tools`:
82-
- `gen_ai.request.frequency_penalty`: Model configuration
83-
- `gen_ai.request.max_tokens`: Model configuration
84-
- `gen_ai.request.model`:
85-
- `gen_ai.request.presence_penalty`: Model configuration
86-
- `gen_ai.request.temperature`: Model configuration
87-
- `gen_ai.request.top_p`: Model configuration
88-
- `gen_ai.tool.description`: Description of the tool executed
89-
- `gen_ai.tool.input`: Input that was given to the executed too. (Example: \{"max":10\})
90-
- `gen_ai.tool.name:`: Name of the tool executed. (Example: "random_number")
91-
- `gen_ai.tool.output`: The output from the tool.
92-
- `gen_ai.tool.type`: The type of the tools. Can be `function`, `extension`, `datastore`
93-
94-
### AI Client Span
144+
- `gen_ai.request.available_tools`:
145+
- `gen_ai.request.frequency_penalty`: Model configuration
146+
- `gen_ai.request.max_tokens`: Model configuration
147+
- `gen_ai.request.model`:
148+
- `gen_ai.request.presence_penalty`: Model configuration
149+
- `gen_ai.request.temperature`: Model configuration
150+
- `gen_ai.request.top_p`: Model configuration
151+
- `gen_ai.tool.description`: Description of the tool executed
152+
- `gen_ai.tool.input`: Input that was given to the executed too. (Example: \{"max":10\})
153+
- `gen_ai.tool.name:`: Name of the tool executed. (Example: "random_number")
154+
- `gen_ai.tool.output`: The output from the tool.
155+
- `gen_ai.tool.type`: The type of the tools. Can be `function`, `extension`, `datastore`
156+
157+
### Attributes of AI Client Span
95158

96159
This span wraps the request to an LLM.
97160

0 commit comments

Comments
 (0)