Skip to content

Commit ead8249

Browse files
authored
[Docs] Add observability section for agentops (#448)
# [Docs] Add observability section for agentops Adds a new section **Observability** under **Advanced Features** for [AgentOps](https://www.agentops.ai/). This is in accordance with this [PR](AgentOps-AI/agentops#541) and should close #445
1 parent 6d066a6 commit ead8249

File tree

2 files changed

+60
-0
lines changed

2 files changed

+60
-0
lines changed
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
---
2+
id: observability
3+
description: Observability
4+
slug: /observability
5+
---
6+
7+
# Observability
8+
9+
[AgentOps](https://www.agentops.ai/) is a platform that helps you to monitor and analyze the behavior and performance of your agents.
10+
TaskWeaver is integrated with AgentOps, so you can use TaskWeaver to generate code and use AgentOps to observe all the events from the planning to the code execution.
11+
12+
## Get Started
13+
14+
TaskWeaver integration with AgentOps is documented [here](https://www.agentops.ai/docs/taskweaver).
15+
16+
To get started, you need an API key from AgentOps by [signing up](https://www.agentops.ai/signup) for a free account. Then you need to create a new project and generate the API key from the project settings.
17+
18+
Once you have the API keys, the following steps will help you to get started.
19+
20+
1. Install the `agentops`` package in your project.
21+
```bash
22+
pip install agentops
23+
```
24+
2. Import the `agentops` package in your project.
25+
```python
26+
import agentops
27+
```
28+
3. Initialize the `agentops` client with your API key.
29+
```python
30+
agentops.init(api_key="your_api_key")
31+
```
32+
33+
:::note
34+
You can also set the tags here to track your sessions in the AgentOps dashboard. By default, the "taskweaver" tag will be added to all the AgentOps sessions for TaskWeaver.
35+
:::
36+
37+
4. Import the TaskWeaver handler `TaskWeaverEventHandler` and set it as the event handler for your TaskWeaver project.
38+
```python
39+
from agentops.providers.taskweaver import TaskWeaverEventHandler
40+
handler = TaskWeaverEventHandler()
41+
```
42+
43+
:::note
44+
There are two ways to set the handler:
45+
- Set the handler using the `session.event_handler.register()` method.
46+
- Set the handler in every `session.send_message` function call in the `event_handler` parameter.
47+
:::
48+
49+
:::warning
50+
If you encounter "stuttering" in the recorded messages from the events, it is because the handler has been set with more than one instance.
51+
:::
52+
53+
5. Now you can run your TaskWeaver project and observe the events in the AgentOps dashboard. When the client is initialized, it will automatically start to track the events and provide a link to the session in the AgentOps dashboard.
54+
55+
:::info
56+
It is important to note that the LLM calls and the other events are tracked together in different modules of AgentOps. By default, all the LLM calls are tracked when the AgentOps client is initialized. However, the other events are only tracked when the `TaskWeaverEventHandler` handler is used in conjunction within the code.
57+
58+
Therefore, event observability is limited to LLM calls when TaskWeaver is used in [Terminal mode](../usage/cmd.md).
59+
:::

website/sidebars.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@ const sidebars = {
149149
'advanced/code_verification',
150150
'advanced/cli_only',
151151
'advanced/telemetry',
152+
'advanced/observability',
152153
'advanced/plugin_only'
153154
],
154155
}

0 commit comments

Comments
 (0)