Skip to content

Commit 1cc92da

Browse files
authored
docs: improve testgen namings (#1489)
1 parent c01a756 commit 1cc92da

File tree

1 file changed

+45
-17
lines changed
  • docs/concepts/test_data_generation

1 file changed

+45
-17
lines changed

docs/concepts/test_data_generation/rag.md

Lines changed: 45 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,64 @@
11
# Testset Generation for RAG
22

3-
In RAG application, when a user interacts through your application to a set of documents the user may ask different types of queries. These queries in terms of a RAG system can be generally classified into two types:
3+
In RAG application, when a user interacts through your application to a set of documents the there can be different patterns of queries that the system can encounter. Let's first understand the different types of queries that can be encountered in RAG application.
44

5-
## Two fundamental query types in RAG
5+
## Query types in RAG
66

77
```mermaid
88
graph TD
9-
A[Queries] --> B[Specific Queries]
10-
A --> C[Abstract Queries]
9+
A[Queries] --> B[Single-Hop Query]
10+
A --> C[Multi-Hop Query]
11+
12+
B --> D1[Specific Query]
13+
14+
B --> E1[Abstract Query]
15+
16+
C --> F1[Specific Query]
17+
18+
C --> G1[Abstract Query]
1119
```
1220

13-
In any RAG application, when an end user interacts with the system, the queries can be broadly classified into two types:
21+
### Single-Hop Query
22+
23+
A single-hop query is a straightforward question that requires retrieving information from a single document or source to provide a relevant answer. It involves only one step to arrive at the answer.
24+
25+
**Example (Specific Query):**
26+
27+
- “What year did Albert Einstein publish the theory of relativity?”
28+
29+
This is a specific, fact-based question that can be answered with a single retrieval from a document containing that information.
30+
31+
**Example (Abstract Query):**
32+
33+
- “How did Einstein’s theory change our understanding of time and space?”
34+
35+
While this query still refers to a single concept (the theory of relativity), it requires a more abstract or interpretive explanation from the source material.
36+
37+
### Multi-Hop Query
38+
39+
A multi-hop query involves multiple steps of reasoning, requiring information from two or more sources. The system must retrieve information from various documents and connect the dots to generate an accurate answer.
40+
41+
**Example (Specific Query):**
42+
43+
- “Which scientist influenced Einstein’s work on relativity, and what theory did they propose?”
1444

15-
- Specific Queries
16-
- Queries directly answerable by referring to single context
17-
- “What is the value of X in Report FY2020 ?”
45+
This requires the system to retrieve information about both the scientist who influenced Einstein and the specific theory, potentially from two different sources.
1846

19-
- Abstract Queries
47+
**Example (Abstract Query):**
2048

21-
- Queries that can only be answered by referring to multiple documents
22-
- “What is the the revenue trend for Company X from FY2020 through FY2023?”
49+
- “How have scientific theories on relativity evolved since Einstein’s original publication?”
2350

51+
This abstract query requires the retrieval of multiple pieces of information over time and across different sources to form a broad, interpretive response about the evolution of the theory.
2452

25-
Synthesizing specific queries is relatively easy as it requires only a single context to generate the query. However, abstract queries require multiple contexts to generate the query.** Now the fundamental question is how select the right set of chunks to generate the abstract queries**. Different types of abstract queries require different types of contexts. For example,
53+
### Specific vs. Abstract Queries in a RAG
2654

27-
- Abstract queries comparing two entities in a specific domain require contexts that contain information about the entities.
28-
- “Compare the revenue growth of Company X and Company Y from FY2020 through FY2023”
29-
- Abstract queries about the a topic discussed in different contexts require contexts that contain information about the topic.
30-
- “What are the different strategies used by companies to increase revenue?”
55+
- **Specific Query:** Focuses on clear, fact-based retrieval. The goal in RAG is to retrieve highly relevant information from one or more documents that directly address the specific question.
56+
57+
- **Abstract Query:** Requires a broader, more interpretive response. In RAG, abstract queries challenge the retrieval system to pull from documents that contain higher-level reasoning, explanations, or opinions, rather than simple facts.
3158

59+
In both single-hop and multi-hop cases, the distinction between specific and abstract queries shapes the retrieval and generation process by determining whether the focus is on precision (specific) or on synthesizing broader ideas (abstract).
3260

33-
To solve this problem, Ragas uses a Knowledge Graph based approach to Test set Generation.
61+
Different types of queries requires different contexts to be synthesize. To solve this problem, Ragas uses a Knowledge Graph based approach to Test set Generation.
3462

3563
## Knowledge Graph Creation
3664

0 commit comments

Comments
 (0)