You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/guides/genai-leveraging-rag/index.md
+18-45Lines changed: 18 additions & 45 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,14 +14,13 @@ params:
14
14
15
15
## Introduction
16
16
17
-
Retrieval-Augmented Generation (RAG) is a powerful framework that enhances large language models (LLMs) by integrating information retrieval from external knowledge sources. This guide focuses on a specialized RAG implementation using graph databases like Neo4j, which excel in managing highly connected, relational data. Unlike traditional RAG setups with vector databases, combining RAG with graph databases offers better context-awareness and relationship-driven insights, making it ideal for domains like recommendation systems, compliance tracking, and social networks.
17
+
Retrieval-Augmented Generation (RAG) is a powerful framework that enhances large language models (LLMs) by integrating information retrieval from external knowledge sources. This guide focuses on a specialized RAG implementation using graph databases like Neo4j, which excel in managing highly connected, relational data. Unlike traditional RAG setups with vector databases, combining RAG with graph databases offers better context-awareness and relationship-driven insights.
18
18
19
19
In this guide, you will:
20
20
21
-
- Understand the benefits of integrating graph databases into a RAG system.
22
-
- Set up a GenAI stack using Docker, including Neo4j and an AI model.
23
-
- Explore a case study demonstrating the system’s effectiveness with real-world queries.
24
-
21
+
* Explore the advantages of integrating graph databases into a RAG framework.
22
+
* Configure a GenAI stack with Docker, incorporating Neo4j and an AI model.
23
+
* Analyze a real-world case study that highlights the effectiveness of this approach for handling specialized queries.
@@ -34,58 +33,37 @@ RAG (Retrieval-Augmented Generation) is a hybrid framework that enhances the cap
34
33
In a Retrieval-Augmented Generation (RAG) system, vector embeddings are used to represent the semantic meaning of text in a way that a machine can understand and process. For instance, the words "dog" and "puppy" will have similar embeddings because they share similar meanings. By integrating these embeddings into the RAG framework, the system can combine the generative power of large language models with the ability to pull in highly relevant, contextually-aware data from external sources.
35
34
36
35
The system operates as follows:
37
-
1.Queries are transformed into vector embeddings to capture semantic meaning.
38
-
2.Relevant information is retrieved from the knowledge base using these embeddings.
39
-
3. The retrieved data is used to augment the input prompt for the LLM.
40
-
4. The LLM generates responses that blend the model's inherent knowledge with the retrieved context.
36
+
1.Questions get turned into mathematical patterns that capture their meaning
37
+
2.These patterns help find matching information in a database
38
+
3. The found information gets added to the original question before passed to LLM
39
+
4. The LLM generates responses that blend the model's inherent knowledge with the this extra information.
41
40
42
41
To hold this vector information in a efficient manner, we need a special type of database.
43
42
44
-
45
43
## Introduction to Graph Databases
46
44
47
45
Graph databases, such as Neo4j, are specifically designed for managing highly connected data. Unlike traditional relational databases, graph databases prioritize both the entities and the relationships between them, making them ideal for tasks where connections are as important as the data itself.
48
46
49
-
In the broader landscape of NoSQL databases, there are four main categories:
50
-
51
-
* Document-based databases
52
-
* Key-value stores
53
-
* Column-oriented databases
54
-
* Graph-based databases
55
-
56
-
As one of these core categories, graph databases stand out for their unique approach to data storage and querying. They use nodes (or vertices) to represent entities and edges to represent the relationships between these entities. This structure allows for efficient handling of highly connected data and complex queries, which are difficult to manage in traditional database systems.
57
-
58
-
### Graph Databases vs SQL Databases
59
-
60
-
1. Data Model:
61
-
- SQL: Uses tables with rows and columns, relationships expressed through foreign keys
62
-
- Graph: Uses nodes and edges, relationships are much more flexible
47
+
Graph databases stand out for their unique approach to data storage and querying. They use nodes (or vertices) to represent entities and edges to represent the relationships between these entities. This structure allows for efficient handling of highly connected data and complex queries, which are difficult to manage in traditional database systems.
63
48
64
-
2. Schema Flexibility:
65
-
- SQL: Rigid schema, changes require extra steps
66
-
- Graph: Flexible schema, can add new relationships without migrations
49
+
SQL databases and graph databases differ significantly in their data models. SQL databases use a tabular structure with rows and columns, where relationships between entities are established using foreign keys. This approach works well for structured data and predefined relationships. In contrast, graph databases represent data as nodes (entities) and edges (relationships), making the representation of relationships more intuitive and flexible. This structure is particularly advantageous for complex, interconnected data.
67
50
68
-
3. Use Cases:
69
-
- SQL: Structured data with fixed relationships
70
-
- Graph: Social networks, recommendation engines, knowledge graphs
51
+
Another key difference lies in schema flexibility. SQL databases operate on a rigid schema, meaning any changes to the data structure, such as adding new columns or altering relationships, typically require careful planning and migration processes. Graph databases, however, are schema-optional, allowing for much greater flexibility. New nodes, edges, or properties can be introduced without disrupting existing data, enabling faster adaptation to changing requirements.
71
52
72
-
73
-
74
-
75
-
## Practical Implementation: A Case Study in RAG Effectiveness
To illustrate the power of RAG systems in practice, let's examine a real-world implementation using Apache NiFi as our subject matter. This case study demonstrates how RAG can enhance an AI's ability to provide accurate, contextual information about specialized technical topics.
78
56
79
-
80
-
81
-
## A Case Study in RAG Implementation
82
-
83
57
### Teaching AI About New Technologies
84
58
85
59
Apache NiFi serves as an excellent example of the limitations of traditional LLMs and how RAG can overcome them. As a relatively recent technology, many LLMs have limited or outdated knowledge about it, making it a perfect candidate for demonstrating RAG's capabilities.
86
60
87
61
Without RAG, the LLM may provide broad responses or acknowledge its limited knowledge when asked about information that is beyond its cutoff date.
88
62
63
+
Question: What important events happened in 2024?
64
+
RAG: Disabled
65
+
66
+
I'm happy to help! Unfortunately, I'm a large language model, I don't have access to real-time information or events that occurred after my training data cutoff in 2024. Therefore, I cannot provide you with any important events that happened in 2024. My apologize for any inconvenience this may cause. Is there anything else I can help you with?
89
67
90
68
## Setting Up GenAI Stack with GPU Acceleration on Linux
91
69
@@ -106,8 +84,6 @@ Make sure following lines are commented out. Set your own credentials for securi
106
84
NEO4J_PASSWORD=password
107
85
OLLAMA_BASE_URL=http://llm-gpu:11434
108
86
109
-
110
-
111
87
### CPU powered
112
88
113
89
```bash
@@ -123,17 +99,17 @@ Make sure following lines are commented out. Set your own credentials for securi
123
99
NEO4J_PASSWORD=password
124
100
OLLAMA_BASE_URL=http://llm:11434
125
101
126
-
127
102
### Setting Up on Other Platforms
128
103
129
104
For instructions on how to set up the stack on other platforms, refer to [this page](https://github.com/docker/genai-stack).
130
105
131
-
---
106
+
132
107
### Initial Startup
133
108
134
109
The first startup may take some time because the system needs to download a large language model (LLM).
135
110
136
111
### Monitoring Progress
112
+
137
113
We can monitor the download and initialization progress by viewing the logs. Run the following command to view the logs:
138
114
139
115
```bash
@@ -151,7 +127,6 @@ Wait for specific lines in the logs indicating that the download is complete and
151
127
152
128
You can now access the interface at [http://localhost:8501/](http://localhost:8501/) to ask questions. For example, you can try the sample question:
153
129
154
-
155
130
When we see those lines in the logs, web apps are ready to be used.
156
131
157
132
Since our goal is to teach AI about things it does not yet know, we begin by asking it a simple question about Nifi at
@@ -173,14 +148,12 @@ Now it's time to teach the AI some new tricks. First, connect to [http://localho
173
148

174
149
175
150
176
-
177
151
After the import is successful, we can access Neo4j to verify the data.
178
152
179
153
After logging in to [http://localhost:7474/](http://localhost:7474/) using the credentials from the `.env` file, you can run queries on Neo4j. Using the Neo4j Cypher query language, you can check for the data stored in the database.
180
154
181
155
To count the data, run the following query:
182
156
183
-
184
157
```cypher
185
158
MATCH (n)
186
159
RETURN DISTINCT labels(n) AS NodeTypes, count(*) AS Count
0 commit comments