You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/explorer/api/annotations.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ Annotations provide additional context, facilitating collaboration and agent err
16
16
17
17
## Annotation Format
18
18
19
-
You can add annotations to traces at upload time. For this, both during [file upload](../uploading-traces/file-uploads) and via the [Push API](../uploading-traces/push-api), you can include an `annotations` field in the trace data. This field should be an array of objects, each representing an annotation. Each annotation object should have the following fields:
19
+
You can add annotations to traces at upload time. For this, both during [file upload](uploading-traces/file-uploads.md) and via the [Push API](uploading-traces/push-api.md), you can include an `annotations` field in the trace data. This field should be an array of objects, each representing an annotation. Each annotation object should have the following fields:
Copy file name to clipboardExpand all lines: docs/explorer/api/uploading-traces/file-uploads.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,7 +20,7 @@ The explorer supports two types of trace formats: _raw event lists_ and _annotat
20
20
21
21
#### Raw Event Lists
22
22
23
-
Raw event lists are `jsonl` files where each line is a JSON array of events. Each event is a dictionary with at least a `role` and `content` field according to the trace format described [in this chapter](../../trace-format).
23
+
Raw event lists are `jsonl` files where each line is a JSON array of events. Each event is a dictionary with at least a `role` and `content` field according to the trace format described [in this chapter](../trace-format.md).
Copy file name to clipboardExpand all lines: docs/explorer/api/uploading-traces/push-api.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,7 +24,7 @@ The `PushTracesRequest` class holds the request data for a trace upload request.
24
24
25
25
This represents the traces in a dataset. Each `List[Dict]` is a single trace within the dataset. Each `dict` is a single message within a trace - these can represet a user prompt, a tool call, a tool output, etc.
26
26
27
-
Must be in the [required trace format](../../trace-format). Must not be empty.
27
+
Must be in the [required trace format](../trace-format.md). Must not be empty.
Copy file name to clipboardExpand all lines: docs/explorer/self-hosted.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,15 +40,15 @@ This will pull and launch the required Explorer Docker containers on your machin
40
40
41
41
## Using the Self-Hosted Explorer
42
42
43
-
The self-hosted version of Explorer is configured to run on `http://localhost`. You can access it in your browser at this address. The local instance provides the same API as the managed cloud instance, so you can use the [Invariant SDK](./api/client-setup) to connect to it.
43
+
The self-hosted version of Explorer is configured to run on `http://localhost`. You can access it in your browser at this address. The local instance provides the same API as the managed cloud instance, so you can use the [Invariant SDK](api/client-setup.md) to connect to it.
44
44
45
45
### Storage
46
46
47
47
The self-hosted version of Explorer will create a `data/` directory in the current working directory to store traces and other data.
48
48
49
49
### Usage and Access
50
50
51
-
You can access the self-hosted version of Explorer at `http://localhost`. To use it with the [Invariant SDK](./api/client-setup) you can set the `INVARIANT_API_ENDPOINT` environment variable to `http://localhost/`. For security reasons, you'll still need to create and provide an API key to access the self-hosted version of Explorer.
51
+
You can access the self-hosted version of Explorer at `http://localhost`. To use it with the [Invariant SDK](api/client-setup.md) you can set the `INVARIANT_API_ENDPOINT` environment variable to `http://localhost/`. For security reasons, you'll still need to create and provide an API key to access the self-hosted version of Explorer.
52
52
53
53
### Updates
54
54
`invariant explorer` will automatically check for updates and pull the latest stable version of the Explorer Docker images. If you want to try the latest development version, you can use the `--version=main` flag. Note however, that database migrations may be required when switching between versions, which may not always be backwards compatible.
Copy file name to clipboardExpand all lines: docs/testing/index.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ title: Overview
6
6
7
7
Invariant `testing` is a lightweight library to write and run AI agent tests. It provides helpers and assertions that enable you to write robust tests for your agentic applications.
8
8
9
-
Using [**localized assertions**](Writing_Tests/1_Traces.ipynb), `testing` always points you to the exact part of the agent's behavior that caused a test to fail, making it easy to debug and resolve issues (_think: stacktraces for agents_).
9
+
Using [**localized assertions**](writing/traces.ipynb), `testing` always points you to the exact part of the agent's behavior that caused a test to fail, making it easy to debug and resolve issues (_think: stacktraces for agents_).
The test result provides information about which assertion failed but also [localizes the assertion failure precisely](writing/tests) in the provided list of agent messages.
85
+
The test result provides information about which assertion failed but also [localizes the assertion failure precisely](writing/tests.md) in the provided list of agent messages.
86
86
87
87
**Visual Test Viewer (Explorer):**
88
88
89
-
As an alternative to the command line, you can also [visualize test results](running/visual-debugging) on the [Invariant Explorer](https://explorer.invariantlabs.ai/):
89
+
As an alternative to the command line, you can also [visualize test results](running/visual-debugging.md) on the [Invariant Explorer](https://explorer.invariantlabs.ai/):
90
90
91
91
```py
92
92
$ invariant test --push
@@ -98,12 +98,12 @@ Like the terminal output, the Explorer highlights the relevant ranges, but does
98
98
99
99
## Features
100
100
101
-
* Comprehensive [`Trace` API](writing/traces) for easily navigating and checking agent traces.
102
-
*[Assertions library](writing/matchers) to check agent behavior, including fuzzy checkers such as _Levenshtein distance_, _semantic similarity_ and _LLM-as-a-judge_ pipelines.
103
-
* Full [`pytest` compatibility](running/pytest-compatibility) for easy integration with existing test and CI/CD pipelines.
104
-
* Parameterized tests for [testing multiple scenarios](writing/parameterized-tests) with a single test function.
105
-
*[Visual test viewer](running/visual-debugging) for exploring large traces and debugging test failures.
101
+
* Comprehensive [`Trace` API](writing/traces.ipynb) for easily navigating and checking agent traces.
102
+
*[Assertions library](writing/matchers.md) to check agent behavior, including fuzzy checkers such as _Levenshtein distance_, _semantic similarity_ and _LLM-as-a-judge_ pipelines.
103
+
* Full [`pytest` compatibility](running/pytest-compatibility.md) for easy integration with existing test and CI/CD pipelines.
104
+
* Parameterized tests for [testing multiple scenarios](writing/parameterized-tests.md) with a single test function.
105
+
*[Visual test viewer](running/visual-debugging.md) for exploring large traces and debugging test failures.
Copy file name to clipboardExpand all lines: docs/testing/running/pytest-compatibility.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@
8
8
9
9
`invariant test` is a lightweight wrapper of the original `pytest` CLI. It is designed to be a drop-in replacement for `pytest` in your existing test suite.
10
10
11
-
On top of the standard `pytest` arguments, `invariant test` supports the following extra arguments to enable [pushing to Explorer](./Visual_Debugger.md).
11
+
On top of the standard `pytest` arguments, `invariant test` supports the following extra arguments to enable [pushing to Explorer](./visual-debugging.md).
Copy file name to clipboardExpand all lines: docs/testing/writing/integrate.md
+1-17Lines changed: 1 addition & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -36,44 +36,28 @@ Each message has a `role` key that specifies the role of the speaker (e.g., `use
36
36
37
37
Based on this simple format, it is easy to integrate your agent with Invariant `testing`.
38
38
39
-
> **Extended Format Support**`testing` also supports additional custom properties in the trace format, e.g. for metadata. Note, however, that convenience methods like `Trace.tool_calls()` assume the standard format. Similarly, if you are planning to visualize the trace in [Explorer](/explorer/), we advise you to stick to the standard format.
39
+
> **Extended Format Support**`testing` also supports additional custom properties in the trace format, e.g. for metadata. Note, however, that convenience methods like `Trace.tool_calls()` assume the standard format. Similarly, if you are planning to visualize the trace in [Explorer](explorer/index.md), we advise you to stick to the standard format.
40
40
41
41
To learn about how to do this, follow one of the example guides below:
0 commit comments