Skip to content

Commit e4cdd30

Browse files
improved reference
1 parent 1522c83 commit e4cdd30

File tree

1 file changed

+86
-1
lines changed

1 file changed

+86
-1
lines changed

docs/guardrails/rules.md

Lines changed: 86 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ icon: bootstrap/book
88
A concise reference for writing guardrailing rules with Invariant.
99
</div>
1010

11+
This page contains a concise reference for writing guardrailing rules with Invariant. For a more guided introduction, please refer to the [introduction chapter](./index.md).
12+
1113
## Setting Up Your LLM Client
1214

1315
To get started with guardrailing, you have to setup your LLM client to use [Invariant Gateway](../gateway/index.md):
@@ -47,6 +49,8 @@ Before you run, make sure you export the relevant environment variables includin
4749

4850
## Message-Level Guardrails
4951

52+
See also [Ban Topics and Substrings](../guardrails/ban-words.md).
53+
5054
**Example:** Checking for the presence of specific keywords in the message content.
5155
```guardrail
5256
raise "The one who must not be named" if:
@@ -87,9 +91,10 @@ raise "Prompt Injection Detected" if:
8791

8892
See also [Jailbreaks and Prompt Injections](../guardrails/prompt-injections.md) and [Moderated Content](../guardrails/moderation.md).
8993

90-
9194
## Tool Call Guardrails
9295

96+
See also [Tool Calls](../guardrails/tool-calls.md) for more details on tool call guardrailing.
97+
9398
**Example**: Matching a `send_email` tool call with a specific recipient.
9499
```guardrail
95100
raise "Must not send any emails to Alice" if:
@@ -204,6 +209,8 @@ raise "PII in tool output" if:
204209

205210
## Code Guardrails
206211

212+
See also [Code Validation](../guardrails/code-validation.md) for more details on code validation guardrailing.
213+
207214
**Example:** Validating the function calls in a code snippet.
208215
```guardrail
209216
from invariant.detectors.code import python_code
@@ -257,6 +264,8 @@ raise "Dangerous pattern detected in about-to-be-executed bash command" if:
257264

258265
## Content Guardrails (PII, Copyright, etc.)
259266

267+
See also [PII Detection](../guardrails/pii.md) and [Content Moderation](../guardrails/moderation.md) for more details on content guardrailing.
268+
260269
**Example:** Detecting any PII in any message.
261270
```guardrail
262271
from invariant.detectors import pii
@@ -451,6 +460,8 @@ raise "found copyrighted code" if:
451460

452461
## Data Flow Guardrails
453462

463+
See also [Data Flow Rules](../guardrails/dataflow-rules.md) for more details on data flow guardrailing.
464+
454465
**Example:** Preventing a simple flow
455466
```guardrail
456467
raise "Must not call tool after user uses keyword" if:
@@ -596,6 +607,8 @@ raise "Must not call tool after user uses keyword" if:
596607

597608
## Loop Detection
598609

610+
See also [Loop Detection](../guardrails/loops.md) for more details on loop detection guardrailing.
611+
599612
**Example:** Limiting the number of calls to a certain tool.
600613
```guardrail
601614
from invariant import count
@@ -657,4 +670,76 @@ raise "Allocated too many virtual machines" if:
657670
"content": "Virtual machine allocated successfully"
658671
}
659672
]
673+
```
674+
675+
**Example:** Detecting a retry loop with quantifiers
676+
677+
```guardrail
678+
from invariant import count
679+
680+
raise "Repetition of length in [2,10]" if:
681+
# start with any check_status tool call
682+
(call1: ToolCall)
683+
call1 is tool:check_status
684+
685+
# there need to be between 2 and 10 other
686+
# calls to the same tool, after 'call1'
687+
count(min=2, max=10):
688+
call1 -> (other_call: ToolCall)
689+
other_call is tool:check_status
690+
```
691+
692+
```example-trace
693+
[
694+
{
695+
"role": "user",
696+
"content": "Reply to Peter's message"
697+
},
698+
{
699+
"role": "assistant",
700+
"content": "",
701+
"tool_calls": [
702+
{
703+
"id": "1",
704+
"type": "function",
705+
"function": {
706+
"name": "check_status",
707+
"arguments": {}
708+
}
709+
}
710+
]
711+
},
712+
{
713+
"role": "assistant",
714+
"content": "There seems to be an issue with the server. I will check the status and get back to you."
715+
},
716+
{
717+
"role": "assistant",
718+
"content": "",
719+
"tool_calls": [
720+
{
721+
"id": "1",
722+
"type": "function",
723+
"function": {
724+
"name": "check_status",
725+
"arguments": {}
726+
}
727+
}
728+
]
729+
},
730+
{
731+
"role": "assistant",
732+
"content": "",
733+
"tool_calls": [
734+
{
735+
"id": "1",
736+
"type": "function",
737+
"function": {
738+
"name": "check_status",
739+
"arguments": {}
740+
}
741+
}
742+
]
743+
}
744+
]
660745
```

0 commit comments

Comments
 (0)