Skip to content

Commit 1f17112

Browse files
remove some placeholders
1 parent 40e7a7c commit 1f17112

File tree

2 files changed

+64
-20
lines changed

2 files changed

+64
-20
lines changed

docs/testing/Writing_Tests/Integrate_Your_Agent.md

Lines changed: 56 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,22 +2,64 @@
22

33
<div class='subtitle'>Get your agent ready for testing</div>
44

5-
Invariant `testing` relies on a very simple yet powerful trace format as the common denominator for testing agents.
5+
`testing` works with a simple yet powerful trace format as the common denominator for testing agents.
66

7-
This trace format is a list of dictionaries, where each dictionary represents a message in the conversation. Each message has a `role` key that specifies the role of the speaker (e.g., `user` or `agent`) and a `content` key that contains the message content.
7+
This trace format is a list of dictionaries, where each dictionary represents a message in the conversation. It is equivalent to the [OpenAI chat format](https://platform.openai.com/docs/api-reference/chat/create).
8+
9+
Each message has a `role` key that specifies the role of the speaker (e.g., `user` or `assistant`) and a `content` key that contains the message content.
810

911
```json
10-
{"role": "user", "content": "Hello there"},
11-
{"role": "assistant", "content": "Hello there", "tool_calls": [
12-
{
13-
"type": "function",
14-
"function": {
15-
"name": "greet",
16-
"arguments": {
17-
"name": "there"
12+
{
13+
"role": "user",
14+
"content": "Hello there"
15+
},
16+
{
17+
"role": "assistant",
18+
"content": "Hello there",
19+
"tool_calls": [
20+
{
21+
"type": "function",
22+
"function": {
23+
"name": "greet",
24+
"arguments": {
25+
"name": "there"
26+
}
1827
}
1928
}
20-
}
21-
]},
22-
{"role": "user", "content": "I need help with something."},
23-
```
29+
]
30+
},
31+
{
32+
"role": "user",
33+
"content": "I need help with something."
34+
}
35+
```
36+
37+
Based on this simple format, it is easy to integrate your agent with Invariant `testing`.
38+
39+
> **Extended Format Support** `testing` also supports additional custom properties in the trace format, e.g. for metadata. Note, however, that convenience methods like `Trace.tool_calls()` assume the standard format. Similarly, if you are planning to visualize the trace in [Explorer](/explorer/), we advise you to stick to the standard format.
40+
41+
To learn about how to do this, follow one of the example guides below:
42+
43+
<div class='tiles'>
44+
45+
<a href="/testing/Examples/computer-use/" class='tile primary'>
46+
<span class='tile-title'>Computer Use Agents →</span>
47+
<span class='tile-description'>Integrate your computer use agent with Invariant <code>testing</code></span>
48+
</a>
49+
50+
<a href="/testing/Examples/langgraph/" class='tile primary'>
51+
<span class='tile-title'>LangGraph Agents →</span>
52+
<span class='tile-description'>Test your LangGraph agent with <code>testing</code></span>
53+
</a>
54+
55+
<a href="/testing/Examples/openai-python-agent/" class='tile'>
56+
<span class='tile-title'>Function Calling Agents →</span>
57+
<span class='tile-description'>Integrate your agent with Invariant <code>testing</code> using function calls</span>
58+
</a>
59+
60+
<a href="/testing/Examples/swarm/" class='tile'>
61+
<span class='tile-title'>Swarm Agents →</span>
62+
<span class='tile-description'>Test your swarm agent with Invariant <code>testing</code></span>
63+
</a>
64+
65+
</div>

docs/testing/Writing_Tests/Matchers.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,18 @@ To accommodate this, `testing` includes several different `Matcher` implementati
88

99
Beyond that, `Matcher` is also a simple base class that allows you to write your own custom matchers, if the provided ones are not sufficient for your needs (e.g. custom properties).
1010

11-
## `IsSimilar`
11+
## [`IsSimilar`](https://github.com/invariantlabs-ai/invariant/blob/main/testing/invariant/custom_types/matchers.py#L53)
1212

13-
TODO
13+
Checks if a string is similar to an expected string by checking if the similary score reaches a given threshold.
1414

15-
## `LambdaMatcher`
15+
## [`LambdaMatcher`](https://github.com/invariantlabs-ai/invariant/blob/main/testing/invariant/custom_types/matchers.py#L18)
1616

17-
TODO
17+
Matcher for checking if a lambda function returns True on the underlying value. This can be useful to check for custom properties of outputs, while maintaining [addresses to localize failing](./tests.md) assertions.
1818

19-
## `IsFactuallyEqual`
19+
## [`IsFactuallyEqual`](https://github.com/invariantlabs-ai/invariant/blob/main/testing/invariant/custom_types/matchers.py#L86)
2020

2121
Checks for factual equality / entailment of two sentences or words. This can be used to check if two sentences are factually equivalent, or subset/superset of each other.
2222

23-
TODO
23+
## [`ContainsImage`](https://github.com/invariantlabs-ai/invariant/blob/main/testing/invariant/custom_types/matchers.py#L153)
24+
25+
Checks if the input contains an image in base64 encoding.

0 commit comments

Comments
 (0)