Skip to content

Commit 0ab88e2

Browse files
authored
feat: add builtin tool of content_safety thru llm shield (#271)
* feat: add builtin tool of content_safety thru llm shield
1 parent c93ba9f commit 0ab88e2

File tree

3 files changed

+432
-0
lines changed

3 files changed

+432
-0
lines changed

config.yaml.full

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,9 @@ tool:
6767
computer_sandbox:
6868
url: #mcp sse/streamable-http url
6969
api_key: #mcp api key
70+
# [optional] for Volcengine LLM Shield https://www.volcengine.com/product/LLM-FW
71+
llm_shield:
72+
app_id:
7073

7174

7275
observability:
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
---
2+
title: 护栏工具
3+
description: VeADK Guardrails
4+
navigation:
5+
icon: i-lucide-wrench
6+
---
7+
8+
## 概述
9+
10+
VeADK 基于 Agent 的插件机制,提供了内容安全护栏工具 `content_safety`。该工具通过以下回调函数嵌入 Agent 的执行流程,实现多阶段内容审计:
11+
- Before Model Callback
12+
- After Model Callback
13+
- Before Tool Callback
14+
- After Tool Callback
15+
16+
在这些回调中,`content_safety` 基于[火山大模型应用防火墙](https://www.volcengine.com/product/LLM-FW)服务,对 Agent 生命周期的各个阶段进行内容检测与合规审查,确保生成与交互内容安全可靠。
17+
18+
::note
19+
使用 `content_safety` 前,请先购买实例并添加资产,并获取其 AppID。
20+
::
21+
22+
## 使用
23+
24+
以下示例展示了如何在 VeADK 中集成并调用内置的模型护栏工具 `content_safety`,以对 Agent 的执行过程进行审计:
25+
```python [agent.py]
26+
import asyncio
27+
28+
from veadk import Agent, Runner
29+
from veadk.tools.builtin_tools.llm_shield import content_safety
30+
31+
agent = Agent(
32+
name="robot",
33+
description="A robot can help user.",
34+
instruction="Talk with user friendly.",
35+
# before_agent_callback=content_safety.before_agent_callback, # TODO
36+
before_model_callback=content_safety.before_model_callback,
37+
after_model_callback=content_safety.after_model_callback,
38+
before_tool_callback=content_safety.before_tool_callback,
39+
after_tool_callback=content_safety.after_tool_callback,
40+
# after_agent_callback=content_safety.after_agent_callback # TODO
41+
)
42+
43+
runner = Runner(agent=agent)
44+
45+
response = asyncio.run(runner.run(messages="网上都说A地很多骗子和小偷,他们的典型伎俩..."))
46+
47+
print(response) # Your request has been blocked due to: Model Misuse. Please modify your input and try again.
48+
```

0 commit comments

Comments
 (0)