Skip to content

Commit ee8d83b

Browse files
cuericleecuericlee
authored
feat: add observability tutorials (#74)
2 parents 8115d75 + ab46691 commit ee8d83b

10 files changed

+679
-0
lines changed

‎01-tutorials/06-observability/01-agentkit-runtime-hosted.ipynb‎

Lines changed: 313 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,313 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "e8828adfe8cce06f",
6+
"metadata": {},
7+
"source": [
8+
"# AgentKit Observability Tutorial 01: Agentkit Runtime 开启可观测\n",
9+
"\n",
10+
"## 概述\n",
11+
"\n",
12+
"本教程面向 **第一次接触 AgentKit SDK 的新手用户**，在一个 Jupyter Notebook 中，带你完成：\n",
13+
" - 对于 在 agentkit runtime 中部署的 Agentit runtime 实例，如何完成可观测数据的接入与上报。\n",
14+
" - 如何查看 Agentkit 可观测数据。\n",
15+
"\n",
16+
"你可以直接从上到下依次运行每个代码单元。建议在 **仓库根目录** 或已安装好 `agentkit` 的 Python 环境中运行。\n",
17+
"\n",
18+
"### Tutorial 说明\n",
19+
"|分类| 详情 |\n",
20+
"|:--------------------|:------------------------------------------------|\n",
21+
"| Agent 类型 | 单Agent |\n",
22+
"| Agent 开发框架 | Volcengine Agent Development Kit(VeADK) |\n",
23+
"| Tutorial 核心模块 | AgentKit Runtime, ApmPlus |\n",
24+
"| 教程复杂度 | 简单 |\n",
25+
"| 使用到的SDK | AgentKit Python SDK, VeADK |\n"
26+
]
27+
},
28+
{
29+
"cell_type": "markdown",
30+
"id": "c1e4ba2cddf8673c",
31+
"metadata": {},
32+
"source": [
33+
"## 教程\n"
34+
]
35+
},
36+
{
37+
"cell_type": "markdown",
38+
"id": "54603202de53e245",
39+
"metadata": {},
40+
"source": [
41+
"### 0. 前置条件\n",
42+
"要执行本教程，您需要：\n",
43+
"1. **火山引擎账号 AK SK**：一个已开通 AgentKit 与 APMPlus 服务的火山引擎账号，并获取其 AccessKey & SecretKey。并将其配置到环境变量中。\n",
44+
" >**提示**：请确保您的 AccessKey 拥有 `AgentKitFullAccess` 与 `APMPlusFullAccess` 权限。\n",
45+
"\n",
46+
"2. **环境准备**：Python 3.10+ 环境准备并安装相关依赖\n",
47+
"\n"
48+
]
49+
},
50+
{
51+
"cell_type": "markdown",
52+
"id": "26c768d1eedbfd3b",
53+
"metadata": {},
54+
"source": [
55+
"### 1. 准备工作\n",
56+
"依赖安装。安装 agentkit sdk 等依赖。"
57+
]
58+
},
59+
{
60+
"cell_type": "code",
61+
"execution_count": null,
62+
"id": "52bddfcc2e788acf",
63+
"metadata": {},
64+
"outputs": [],
65+
"source": [
66+
"!pip install -r requirements.txt"
67+
]
68+
},
69+
{
70+
"cell_type": "markdown",
71+
"id": "e1e5e40c23294342",
72+
"metadata": {},
73+
"source": [
74+
"### 2. 初始化 agentkit 项目\n",
75+
"初始化一个 agentkit 项目，可基于[AgentKit Runtime 教程](https://volcengine.github.io/agentkit-sdk-python/content/4.runtime/1.runtime_quickstart.html) 创建一个 Agent。可通过执行以下命令初始化项目："
76+
]
77+
},
78+
{
79+
"cell_type": "code",
80+
"execution_count": null,
81+
"id": "0d8d2ae9",
82+
"metadata": {},
83+
"outputs": [],
84+
"source": [
85+
"!agentkit init"
86+
]
87+
},
88+
{
89+
"cell_type": "markdown",
90+
"id": "152a8874",
91+
"metadata": {},
92+
"source": [
93+
"以下是通过 agentkit v0.1.15 版本创建出的 agent 代码："
94+
]
95+
},
96+
{
97+
"cell_type": "code",
98+
"execution_count": null,
99+
"id": "443caa039b7e4f33",
100+
"metadata": {},
101+
"outputs": [],
102+
"source": [
103+
"# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.\n",
104+
"#\n",
105+
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
106+
"# you may not use this file except in compliance with the License.\n",
107+
"# You may obtain a copy of the License at\n",
108+
"#\n",
109+
"# http://www.apache.org/licenses/LICENSE-2.0\n",
110+
"#\n",
111+
"# Unless required by applicable law or agreed to in writing, software\n",
112+
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
113+
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
114+
"# See the License for the specific language governing permissions and\n",
115+
"# limitations under the License.\n",
116+
"\n",
117+
"import logging\n",
118+
"\n",
119+
"from veadk import Agent, Runner\n",
120+
"\n",
121+
"from agentkit.apps import AgentkitSimpleApp\n",
122+
"from veadk.prompts.agent_default_prompt import DEFAULT_DESCRIPTION, DEFAULT_INSTRUCTION\n",
123+
"\n",
124+
"logger = logging.getLogger(__name__)\n",
125+
"logger.setLevel(logging.INFO)\n",
126+
"\n",
127+
"\n",
128+
"app = AgentkitSimpleApp()\n",
129+
"\n",
130+
"agent_name = \"Agent\"\n",
131+
"description = DEFAULT_DESCRIPTION \n",
132+
"system_prompt = DEFAULT_INSTRUCTION \n",
133+
"\n",
134+
"\n",
135+
"tools = []\n",
136+
"\n",
137+
"# from veadk.tools.builtin_tools.web_search import web_search\n",
138+
"# tools.append(web_search)\n",
139+
"\n",
140+
"\n",
141+
"agent = Agent(\n",
142+
" name=agent_name,\n",
143+
" description=description,\n",
144+
" instruction=system_prompt,\n",
145+
" tools=tools,\n",
146+
")\n",
147+
"runner = Runner(agent=agent)\n",
148+
"\n",
149+
"\n",
150+
"@app.entrypoint\n",
151+
"async def run(payload: dict, headers: dict) -> str:\n",
152+
" prompt = payload[\"prompt\"]\n",
153+
" user_id = headers[\"user_id\"]\n",
154+
" session_id = headers[\"session_id\"]\n",
155+
"\n",
156+
" logger.info(\n",
157+
" f\"Running agent with prompt: {prompt}, user_id: {user_id}, session_id: {session_id}\"\n",
158+
" )\n",
159+
" response = await runner.run(messages=prompt, user_id=user_id, session_id=session_id)\n",
160+
"\n",
161+
" logger.info(f\"Run response: {response}\")\n",
162+
" return response\n",
163+
"\n",
164+
"\n",
165+
"@app.ping\n",
166+
"def ping() -> str:\n",
167+
" return \"pong!\"\n",
168+
"\n",
169+
"\n",
170+
"if __name__ == \"__main__\":\n",
171+
" app.run(host=\"0.0.0.0\", port=8000)\n"
172+
]
173+
},
174+
{
175+
"cell_type": "markdown",
176+
"id": "9edb9d8499ad4e4",
177+
"metadata": {},
178+
"source": [
179+
"### 3. 开启 APMPlus 可观测\n",
180+
"\n",
181+
"对于在 agentkit runtime 中部署的 Agentit runtime 实例，您不需要进行任何代码修改，只需要在部署时选择开启 `应用监控` 即可自动开启可观测数据的埋点与上报。\n",
182+
"对于通过 agentkit client 进行部署的，默认会开启 `应用监控`。\n",
183+
"\n"
184+
]
185+
},
186+
{
187+
"cell_type": "markdown",
188+
"id": "b59c0090b5753f3d",
189+
"metadata": {},
190+
"source": [
191+
"### 4. 部署 Agent Runtime 实例\n",
192+
"在完成 Agent 代码编写后，您可以使用 agentkit launch 命令一键完成应用的构建和云端部署。该命令会使用 agentkit.yaml 文件中的配置。"
193+
]
194+
},
195+
{
196+
"cell_type": "code",
197+
"execution_count": null,
198+
"id": "2ba2c0dd",
199+
"metadata": {},
200+
"outputs": [],
201+
"source": [
202+
"!agentkit launch"
203+
]
204+
},
205+
{
206+
"cell_type": "markdown",
207+
"id": "863335a2ae6f627d",
208+
"metadata": {},
209+
"source": [
210+
"### 5. 查看 Agentkit 观测数据\n",
211+
"完成 Agent Runtime 部署后，可以通过 agentkit client 进行调用。\n"
212+
]
213+
},
214+
{
215+
"cell_type": "code",
216+
"execution_count": null,
217+
"id": "4842a5d3",
218+
"metadata": {},
219+
"outputs": [],
220+
"source": [
221+
"!agentkit invoke 今天天气怎么样"
222+
]
223+
},
224+
{
225+
"cell_type": "markdown",
226+
"id": "cad3a973",
227+
"metadata": {},
228+
"source": [
229+
"调用完成后，可以在 Agentkit 控制台的[可观测](https://console.volcengine.com/agentkit/region:agentkit+cn-beijing/apmplus)界面查看可观测数据。包括 runtime 监控，模型监控，会话分析等功能。详情参考 [Agentkit 数据观测](https://www.volcengine.com/docs/86681/1873528?lang=zh)。以下为部分功能的效果图：\n",
230+
"\n",
231+
"#### Runtime 监控\n",
232+
"\n",
233+
"![Agentkit 监控面板截图](./images/observability-agentkit-overview.jpg)\n",
234+
"*图1: Agentkit Runtime 概览*\n",
235+
"\n",
236+
"![模型性能监控](./images/observability-agentkit-runtime.jpg)\n",
237+
"*图2: Agentkit Runtime 详情*\n",
238+
"\n",
239+
"#### 模型监控\n",
240+
"![模型性能监控](./images/observability-agentkit-model.jpg)\n",
241+
"*图3: Agentkit 模型监控*\n",
242+
"\n",
243+
"#### 会话分析\n",
244+
"![会话分析界面](./images/observability-agentkit-session.jpg)\n",
245+
"*图4: Agentkit 会话分析*\n",
246+
"\n",
247+
"![会话详情](./images/observability-agentkit-session-detail.jpg)\n",
248+
"*图5: Agentkit 会话详情*\n",
249+
"\n",
250+
"#### Trace分析\n",
251+
"![Trace 分析](./images/observability-agentkit-trace.jpg)\n",
252+
"*图6: Agentkit Trace 分析*\n",
253+
"\n",
254+
"![Trace 详情](./images/observability-agentkit-trace-detail.jpg)\n",
255+
"*图7: Agentkit Trace 详情*\n",
256+
"\n",
257+
"## 总结\n",
258+
"\n",
259+
"本教程带领您完成了在 AgentKit Runtime 中开启可观测性的完整流程，主要内容包括：\n",
260+
"\n",
261+
"1. **可观测性快速接入**：\n",
262+
" - 无需修改任何 Agent 代码，仅需在部署时开启 \"应用监控\" 选项，即可实现可观测数据的自动埋点与上报\n",
263+
"\n",
264+
"2. **完整操作流程**：\n",
265+
" - 安装 AgentKit 依赖\n",
266+
" - 使用 `agentkit init` 初始化 Agent 项目\n",
267+
" - 编写基础 Agent 逻辑（使用默认模板即可）\n",
268+
" - 通过 `agentkit launch` 一键部署到 AgentKit Runtime\n",
269+
" - 使用 `agentkit invoke` 测试调用 Agent\n",
270+
"\n",
271+
"3. **可观测功能介绍**：\n",
272+
" - **Runtime 监控**：查看 Agent 运行状态、请求量、响应时间等基础指标\n",
273+
" - **模型监控**：分析模型调用性能、成本等关键数据\n",
274+
" - **会话分析**：深入了解用户与 Agent 的交互过程和对话历史\n",
275+
" - **Trace 分析**：通过分布式追踪技术，详细查看 Agent 执行流程中的每个环节\n",
276+
"\n",
277+
"\n",
278+
"通过本教程，您已经掌握了 AgentKit Runtime 可观测性的基本配置和使用方法。如需更深入了解相关功能，可以参考 [Agentkit 数据观测](https://www.volcengine.com/docs/86681/1873528?lang=zh) 官方文档。\n",
279+
"\n",
280+
"您也可以扩展此示例的 Agent 以实现更加复杂的 Agent 的可观测性。"
281+
]
282+
},
283+
{
284+
"metadata": {},
285+
"cell_type": "code",
286+
"outputs": [],
287+
"execution_count": null,
288+
"source": "",
289+
"id": "3bce5c79823e9897"
290+
}
291+
],
292+
"metadata": {
293+
"kernelspec": {
294+
"display_name": "Python 3",
295+
"language": "python",
296+
"name": "python3"
297+
},
298+
"language_info": {
299+
"codemirror_mode": {
300+
"name": "ipython",
301+
"version": 2
302+
},
303+
"file_extension": ".py",
304+
"mimetype": "text/x-python",
305+
"name": "python",
306+
"nbconvert_exporter": "python",
307+
"pygments_lexer": "ipython2",
308+
"version": "2.7.6"
309+
}
310+
},
311+
"nbformat": 4,
312+
"nbformat_minor": 5
313+
}

0 commit comments

Comments
 (0)