Skip to content

Commit 9b47c50

Browse files
OhYeeCopilot
andauthored
feat(model): introduce rate limiting and guardrail models with typed … (#19)
* feat(model): introduce rate limiting and guardrail models with typed policy configurations enhance proxy configuration with structured models for token rate limiting and AI guardrails, transition policy definitions from dictionary to typed objects, and extend model proxy props with execution role support; upgrade alibabacloud-agentrun dependency for improved capabilities 为代理配置引入结构化的令牌速率限制和 AI 防护模型,将策略定义从字典类型转为强类型对象, 并在模型代理属性中新增执行角色支持;升级 alibabacloud-agentrun 依赖以提升功能能力 Change-Id: Id35290e04cfb9c4f3f73933febe0d80ff65651a8 Signed-off-by: OhYee <[email protected]> * Update agentrun/model/model.py Co-authored-by: Copilot <[email protected]> Signed-off-by: OhYee <[email protected]> * Update agentrun/model/model.py Co-authored-by: Copilot <[email protected]> Signed-off-by: OhYee <[email protected]> --------- Signed-off-by: OhYee <[email protected]> Co-authored-by: Copilot <[email protected]>
1 parent 9774103 commit 9b47c50

File tree

2 files changed

+19
-3
lines changed

2 files changed

+19
-3
lines changed

agentrun/model/model.py

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -122,18 +122,33 @@ class ProxyConfigFallback(BaseModel):
122122
model_service_name: Optional[str] = None
123123

124124

125+
class ProxyConfigTokenRateLimiter(BaseModel):
126+
tps: Optional[int] = None
127+
tpm: Optional[int] = None
128+
tph: Optional[int] = None
129+
tpd: Optional[int] = None
130+
131+
132+
class ProxyConfigAIGuardrailConfig(BaseModel):
133+
"""AI 防护配置"""
134+
check_request: Optional[bool] = None
135+
check_response: Optional[bool] = None
136+
137+
125138
class ProxyConfigPolicies(BaseModel):
126139
cache: Optional[bool] = None
127140
concurrency_limit: Optional[int] = None
128-
fallbacks: Optional[ProxyConfigFallback] = None
141+
fallbacks: Optional[List[ProxyConfigFallback]] = None
129142
num_retries: Optional[int] = None
130143
request_timeout: Optional[int] = None
144+
ai_guardrail_config: Optional[ProxyConfigAIGuardrailConfig] = None
145+
token_rate_limiter: Optional[ProxyConfigTokenRateLimiter] = None
131146

132147

133148
class ProxyConfig(BaseModel):
134149
endpoints: Optional[List[ProxyConfigEndpoint]] = None
135150
"""代理端点列表"""
136-
policies: Optional[Dict[str, Any]] = None
151+
policies: Optional[ProxyConfigPolicies] = None
137152

138153

139154
class CommonModelMutableProps(BaseModel):
@@ -174,6 +189,7 @@ class ModelProxyMutableProps(CommonModelMutableProps):
174189
proxy_mode: Optional[ProxyMode] = None
175190
service_region_id: Optional[str] = None
176191
proxy_config: Optional[ProxyConfig] = None
192+
execution_role_arn: Optional[str] = None
177193

178194

179195
class ModelProxyImmutableProps(CommonModelImmutableProps):

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ dependencies = [
1313
"litellm>=1.79.3",
1414
"alibabacloud-devs20230714>=2.4.1",
1515
"pydash>=8.0.5",
16-
"alibabacloud-agentrun20250910>=4.0.1",
16+
"alibabacloud-agentrun20250910>=4.0.3",
1717
"alibabacloud_tea_openapi>=0.4.2",
1818
]
1919

0 commit comments

Comments
 (0)