You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/docs/guides/agent/agent_component_reference/retrieval.mdx
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -133,6 +133,10 @@ Before enabling this feature, ensure you have properly [constructed a knowledge
133
133
134
134
Whether to use knowledge graph(s) in the specified dataset(s) during retrieval for multi-hop question answering. When enabled, this would involve iterative searches across entity, relationship, and community report chunks, greatly increasing retrieval time.
135
135
136
+
### PageIndex
137
+
138
+
Whether to use the page index structure generated by the large model to enhance retrieval. This approach mimics human information-searching behavior in books.
139
+
136
140
### Output
137
141
138
142
The global variable name for the output of the **Retrieval** component, which can be referenced by other components in the workflow.
Copy file name to clipboardExpand all lines: website/docs/guides/dataset/extract_table_of_contents.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,14 +7,14 @@ sidebar_custom_props: {
7
7
---
8
8
# Extract table of contents
9
9
10
-
Extract table of contents (TOC) from documents to provide long context RAG and improve retrieval.
10
+
Extract PageIndex, namely table of contents, from documents to provide long context RAG and improve retrieval.
11
11
12
12
---
13
13
14
-
During indexing, this technique uses LLM to extract and generate chapter information, which is added to each chunk to provide sufficient global context. At the retrieval stage, it first uses the chunks matched by search, then supplements missing chunks based on the table of contents structure. This addresses issues caused by chunk fragmentation and insufficient context, improving answer quality.
14
+
During indexing, this technique uses LLM to extract and generate chapter information, which is added to each chunk to provide sufficient global context. At the retrieval stage, it first uses the chunks matched by search, then supplements missing chunks based on the PageIndex (table of contents) structure. This addresses issues caused by chunk fragmentation and insufficient context, improving answer quality.
15
15
16
16
:::danger WARNING
17
-
Enabling TOC extraction requires significant memory, computational resources, and tokens.
17
+
Enabling PageIndex extraction requires significant memory, computational resources, and tokens.
18
18
:::
19
19
20
20
## Prerequisites
@@ -27,15 +27,15 @@ The system's default chat model is used to summarize clustered content. Before p
27
27
28
28
1. Navigate to the **Configuration** page.
29
29
30
-
2. Enable **TOC Enhance**.
30
+
2. Enable **PageIndex**.
31
31
32
32
3. To use this technique during retrieval, do either of the following:
33
33
34
-
- In the **Chat setting** panel of your chat app, switch on the **TOC Enhance** toggle.
35
-
- If you are using an agent, click the **Retrieval** agent component to specify the dataset(s) and switch on the **TOC Enhance** toggle.
34
+
- In the **Chat setting** panel of your chat app, switch on the **PageIndex** toggle.
35
+
- If you are using an agent, click the **Retrieval** agent component to specify the dataset(s) and switch on the **Page Index** toggle.
36
36
37
37
## Frequently asked questions
38
38
39
-
### Will previously parsed files be searched using the TOC enhancement feature once I enable `TOC Enhance`?
39
+
### Will previously parsed files be searched using the directory enhancement feature once I enable `PageIndex`?
40
40
41
-
No. Only files parsed after you enable **TOC Enhance** will be searched using the TOC enhancement feature. To apply this feature to files parsed before enabling **TOC Enhance**, you must reparse them.
41
+
No. Only files parsed after you enable **PageIndex** will be searched using the directory enhancement feature. To apply this feature to files parsed before enabling **PageIndex**, you must reparse them.
0 commit comments