You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/content/docs/autorag/configuration/metadata-filtering.mdx
+14-8Lines changed: 14 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,6 +5,8 @@ sidebar:
5
5
order: 6
6
6
---
7
7
8
+
import { FileTree } from"~/components"
9
+
8
10
Metadata filtering narrows down search results based on metadata, so only relevant content is retrieved. The filter narrows down results prior to retrieval, so that you only query the scope of documents that matter.
9
11
10
12
Here is an example of metadata filtering using [Workers Binding](/autorag/usage/workers-binding/) but it can be easily adapted to use the [REST API](/autorag/usage/rest-api/) instead.
@@ -96,12 +98,16 @@ Note the following limitations with the compound operators:
96
98
You can use "starts with" filtering on the `folder` metadata attribute to search for all files and subfolders within a specific path.
97
99
98
100
For example, consider this file structure:
99
-
```
100
-
customer-a/profile.md
101
-
customer-a/contracts/property/contract-1.pdf
102
-
```
103
101
104
-
If you were to filter using an `eq` (equals) operator with `value: "customer-a/"`, it would only match files directly within that folder, like `profile.md`. It wouldn't include files in subfolders like `customer-a/contracts/`.
102
+
<FileTree>
103
+
- customer-a
104
+
- profile.md
105
+
- contracts
106
+
- property
107
+
- contract-1.pdf
108
+
</FileTree>
109
+
110
+
If you were to filter using an `eq` (equals) operator with `value: "customer-a/"`, it would only match files directly within that folder, like `profile.md`. It would not include files in subfolders like `customer-a/contracts/`.
105
111
106
112
To recursively filter for all items starting with the path `customer-a/`, you can use the following compound filter:
107
113
@@ -117,7 +123,7 @@ filters: {
117
123
{
118
124
type:"lte",
119
125
key:"folder",
120
-
value:"customer-a/z",
126
+
value:"customer-a/z",
121
127
},
122
128
],
123
129
},
@@ -126,8 +132,8 @@ filters: {
126
132
This filter identifies paths starting with `customer-a/` by using:
127
133
128
134
- The `and` condition to combine the effects of the `gt` and `lte` conditions.
129
-
- The `gt` condition to include pathes greater than the `/` ASCII character.
130
-
- The `lte` condition to include pathes less than and including the lower case `z` ASCII character.
135
+
- The `gt` condition to include paths greater than the `/` ASCII character.
136
+
- The `lte` condition to include paths less than and including the lower case `z` ASCII character.
131
137
132
138
Together, these conditions effectively select paths that begin with the provided path value.
Copy file name to clipboardExpand all lines: src/content/docs/autorag/how-to/multitenancy.mdx
+20-13Lines changed: 20 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,6 +5,8 @@ sidebar:
5
5
order: 5
6
6
---
7
7
8
+
import { FileTree } from"~/components"
9
+
8
10
AutoRAG supports multitenancy by letting you segment content by tenant, so each user, customer, or workspace can only access their own data. This is typically done by organizing documents into per-tenant folders and applying [metadata filters](/autorag/configuration/metadata-filtering/) at query time.
9
11
10
12
## 1. Organize Content by Tenant
@@ -13,11 +15,13 @@ When uploading files to R2, structure your content by tenant using unique folder
13
15
14
16
Example folder structure:
15
17
16
-
```bash
17
-
customer-a/logs/
18
-
customer-a/contracts/
19
-
customer-b/contracts/
20
-
```
18
+
<FileTree>
19
+
- customer-a
20
+
- logs/
21
+
- contracts/
22
+
- customer-b
23
+
- contracts/
24
+
</FileTree>
21
25
22
26
When indexing, AutoRAG will automatically store the folder path as metadata under the `folder` attribute. It is recommended to enforce folder separation during upload or indexing to prevent accidental data access across tenants.
23
27
@@ -44,10 +48,13 @@ To filter across multiple folders, or to add date-based filtering, you can use a
44
48
45
49
While an `eq` filter targets files at the specific folder, you'll often want to retrieve all documents belonging to a tenant regardless if there are files in its subfolders. For example, all files in `customer-a/` with a structure like:
46
50
47
-
```
48
-
customer-a/profile.md
49
-
customer-a/contracts/property/contract-1.pdf
50
-
```
51
+
<FileTree>
52
+
- customer-a
53
+
- profile.md
54
+
- contracts
55
+
- property
56
+
- contract-1.pdf
57
+
</FileTree>
51
58
52
59
To achieve this [starts with](/autorag/configuration/metadata-filtering/#starts-with-filter-for-folders) behavior, use a compound filter like:
53
60
@@ -63,7 +70,7 @@ filters: {
63
70
{
64
71
type:"lte",
65
72
key:"folder",
66
-
value:"customer-a/z",
73
+
value:"customer-a/z",
67
74
},
68
75
],
69
76
},
@@ -72,7 +79,7 @@ filters: {
72
79
This filter identifies paths starting with `customer-a/` by using:
73
80
74
81
- The `and` condition to combine the effects of the `gt` and `lte` conditions.
75
-
- The `gt` condition to include pathes greater than the `/` ASCII character.
76
-
- The `lte` condition to include pathes less than and including the lower case `z` ASCII character.
82
+
- The `gt` condition to include paths greater than the `/` ASCII character.
83
+
- The `lte` condition to include paths less than and including the lower case `z` ASCII character.
77
84
78
-
With this filter you would capture both files `profile.md` and `contract-1.pdf`.
85
+
This filter captures both files `profile.md` and `contract-1.pdf`.
0 commit comments