Skip to content

Commit 63f64f7

Browse files
committed
[AIG] Streaming clarification
1 parent 833abdc commit 63f64f7

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/content/docs/ai-gateway/guardrails/usage-considerations.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Since Guardrails runs on Workers AI, enabling it incurs usage on Workers AI. You
1515
- **Latency impact**: Enabling Guardrails adds some latency. Enabling Guardrails introduces additional latency to requests. Typically, evaluations using Llama Guard 3 8B on Workers AI add approximately 500 milliseconds per request. However, larger requests may experience increased latency, though this increase is not linear. Consider this when balancing safety and performance.
1616
- **Handling long content**: When evaluating long prompts or responses, Guardrails automatically segments the content into smaller chunks, processing each through separate Guardrail requests. This approach ensures comprehensive moderation but may result in increased latency for longer inputs.
1717
- **Supported languages**: Llama Guard 3.3 8B supports content safety classification in the following languages: English, French, German, Hindi, Italian, Portuguese, Spanish, and Thai.
18-
18+
- **Streaming support**: Streaming is not supported when using Guardrails.
1919

2020
:::note
2121

0 commit comments

Comments
 (0)