Skip to content

Commit 9107719

Browse files
authored
Update usage-considerations.mdx
add info on expected latency from enabling guardrails
1 parent d614346 commit 9107719

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/content/docs/ai-gateway/guardrails/usage-considerations.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Since Guardrails runs on Workers AI, enabling it incurs usage on Workers AI. You
1212
## Additional considerations
1313

1414
- Model availability: If at least one hazard category is set to `block`, but AI Gateway is unable to receive a response from Workers AI, the request will be blocked. Conversely, if a hazard category is set to `flag` and AI Gateway cannot obtain a response from Workers AI, the request will proceed without evaluation. This approach prioritizes availability, allowing requests to continue even when content evaluation is not possible.
15-
- Latency impact: Enabling Guardrails adds some latency. Consider this when balancing safety and speed.
15+
- Latency impact: Enabling Guardrails adds some latency. Enabling Guardrails introduces additional latency to requests. Typically, evaluations using Llama Guard 3 8B on Workers AI add approximately 500 milliseconds per request. However, larger requests may experience increased latency, though this increase is not linear. Consider this when balancing safety and performance.
1616

1717
:::note
1818

0 commit comments

Comments
 (0)