|
1 | 1 | --- |
2 | | -title: "RAG Example Implementation in the GenAI Showcase App" |
| 2 | +title: "RAG in a Mendix App" |
3 | 3 | url: /appstore/modules/genai/rag/ |
4 | 4 |
|
5 | 5 | linktitle: "Retrieval Augmented Generation (RAG)" |
6 | 6 | weight: 30 |
7 | | -description: "Describes the retrieval augmented generation (RAG) example implementation in the GenAI Showcase App" |
| 7 | +description: "Describes the retrieval augmented generation (RAG) pattern and the example implementation in the GenAI Showcase App" |
8 | 8 | --- |
9 | 9 |
|
10 | 10 | ## Introduction {#introduction} |
11 | 11 |
|
12 | | -Retrieval augmented generation (RAG) is a framework for an AI-based search with a private or external knowledge base that combines embeddings-based knowledge retrieval with a text generation model. The starting point will be a collection of data to be considered as the private knowledge base. The final goal is that an end user of the app can ask questions about the data and the assistant responses will only be based on this knowledge base. |
| 12 | +Retrieval augmented generation (RAG) is a framework for an AI-based search using a private or external knowledge base that combines embeddings-based knowledge retrieval with a text generation model. The starting point is a collection of data to be considered as the private knowledge base. The final goal is that an end user of the app can ask questions about the data and the assistant's responses are only be based on this knowledge base. |
13 | 13 |
|
14 | 14 | {{% alert color="info" %}}This document describes how to set up RAG with PgVector. If you want to use the Bedrock Retrieval Augmented Generation capabilities, see [Bedrock Retrieval Augmented Generation](/appstore/modules/genai/using-gen-ai/#rag).{{% /alert %}} |
15 | 15 |
|
| 16 | +### Terminology |
| 17 | + |
| 18 | +To understand the basics of the RAG pattern, it is important to know the common terminology. As the [showcase example](https://marketplace.mendix.com/link/component/220475) and the relevant platform-supported modules depend on [GenAI Commons](/appstore/modules/genai/commons/), relevant entities will be linked for reference. |
| 19 | + |
| 20 | +#### Embedding vector |
| 21 | + |
| 22 | +Also called **Embedding** and sometimes shortened to **Vector**, this is a mathematical representation of an input string generated by the LLM of choice. It consists of an ordered set of numbers (typically written as [ 0.006, 0.108, ...]), and the total number of elements is called the **dimension**. An embeddings model can convert any string into a vector of fixed dimension. |
| 23 | + |
| 24 | +Every LLM will have its algorithm for generating vectors, but the convention is that conceptually similar strings result in similar vectors. This enables **similarity search** where strings can be matched to a given search string input in terms of semantic meaning (i.e. content/tone/style/...) instead of exact character matches. Minimizing the **cosine distance** between each element and the vector representation of the search string input is a common mathematical technique to search through a collection of vectors and to find the most similar elements. |
| 25 | + |
| 26 | +#### Chunk |
| 27 | + |
| 28 | +In the context of GenAI Commons in a Mendix app, embedding vectors are generated using a [Chunk](/appstore/modules/genai/commons/#chunk-entity). Each object represents a discrete piece of information and contains its original string representation, as well as (after the embedding operation) the vector representation of that string according to the LLM of choice. |
| 29 | + |
| 30 | +#### Knowledge base |
| 31 | + |
| 32 | +This is the place to store discrete pieces of information. If information and its vector representation are stored together, a knowledge base can also be called a **vector database**. Common vector databases have built-in logic to execute similarity searches based on a search vector. |
| 33 | + |
| 34 | +In the context of GenAI Commons in a Mendix app, we use the [PgVector Knowledge Base](https://marketplace.mendix.com/link/component/225063) module to store and retrieve vectors. |
| 35 | + |
| 36 | +#### Knowledge base chunk |
| 37 | + |
| 38 | +In most use cases, more information needs to be stored than just the original input string and its vector representation. A [KnowledgeBaseChunk](/appstore/modules/genai/commons/#knowledgebasechunk-entity) is an extension of [Chunk](/appstore/modules/genai/commons/#chunk-entity) that can hold additional information that is typically required for useful insertion and retrieval from a Mendix application. |
| 39 | + |
| 40 | +#### Metadata |
| 41 | + |
| 42 | +If additional conventional filtering is needed during similarity searches, such additional data can be stored in the knowledge base as well. [Metadata](/appstore/modules/genai/commons/#metadata-entity) objects are key-value pairs that are inserted along with the chunks and contain this additional information. The filtering is applied on an exact string-match basis for the key-value pair. Records are only retrieved if they match all records of the metadata in the collection provided as part of the search step. |
| 43 | + |
| 44 | +{{% alert color="info" %}}The example described in the remainder of this document does not include the more advanced use case of metadata filtering nor does it cover the construction of complex input strings. If you want to see how this can work in practice, take a look at the *RAG with Semantic Search on Historical Data* example in the [GenAI Showcase app](https://marketplace.mendix.com/link/component/220475). {{% /alert %}} |
| 45 | + |
16 | 46 | ## High-level Flow {#rag-high-level} |
17 | 47 |
|
18 | 48 | The complete technical flow can be split up into the following three steps at a high level: |
19 | 49 |
|
20 | 50 | 1. Prepare the knowledge base (once per document) |
21 | 51 | 1. Data is chunked into smaller, partially overlapping, pieces of information. |
22 | | - 2. For each data chunk, the embedding vector will be retrieved from OpenAI's embeddings API. |
| 52 | + 2. For each data chunk, the embedding vector will be retrieved from the LLM's embeddings operations. |
23 | 53 | 3. Data chunks (or their identifier) are stored in a vector database together with their embedding vector. |
24 | 54 |
|
25 | 55 | 2. Query the knowledge base (once per search) |
26 | 56 | 1. User query is sent to the embeddings API to retrieve the embedding vector of the query. |
27 | 57 | 2. A pre-defined number of most-relevant data chunks is retrieved from the vector database. This set is selected based on cosine similarity to the user query embedding vector. |
28 | 58 |
|
29 | 59 | 3. Invoke the text generation model (once per search) |
30 | | - 1. User query and the relevant data chunks are sent to the chat completions API. |
| 60 | + 1. User query and the relevant data chunks are sent to the LLM's chat completions operation. |
31 | 61 | 2. Through prompt engineering, the text generation model is instructed to only base the answer on the data chunks that were sent as part of the request. This prevents the model from hallucinating. |
32 | 62 | 3. The assistant response is returned to the user. |
33 | 63 |
|
34 | | -In summary, in the first step, you need to provide the private knowledge base, such as a text snippet. You need to prepare the content for RAG, which happens only once. If the content changes, you need to provide it again for RAG. The last two steps happen every time an end-user triggers the RAG flow, for example, by asking a question about the data. |
| 64 | +In summary, in the first step, you need to provide the private knowledge base, such as a text snippet. You need to prepare the content for RAG, which happens only once. If the content changes, you need to update the data in the knowledge base. The last two steps happen every time an end-user triggers the RAG flow, for example, by asking a question about the data. |
35 | 65 |
|
36 | 66 | ## RAG Example in the GenAI Showcase App {#rag-showcase-app} |
37 | 67 |
|
38 | 68 | ### Prerequisites {#prerequisites} |
39 | 69 |
|
40 | | -Before you start experimenting with the end-to-end process, make sure that you have covered the following prerequisites: |
| 70 | +Before you start experimenting with the end-to-end process, make sure that you have access to a (remote) PostgreSQL database with the [pgvector](https://github.com/pgvector/pgvector) extension available. If you do not have one yet, [learn more](/appstore/modules/genai/pgvector-setup/) about how a PostgreSQL vector database can be set up to explore use cases with knowledge bases. |
41 | 71 |
|
42 | | -You have access to a (remote) PostgreSQL database with the [pgvector](https://github.com/pgvector/pgvector) extension available. |
43 | | - |
44 | | -{{% alert color="info" %}}If you have access to an Amazon Web Services (AWS) account, Mendix recommends you use a [free-tier RDS](https://aws.amazon.com/rds/faqs/#product-faqs#amazon-rds-faqs#free-tier) setup described in the [Creating a PostgreSQL Database with Amazon RDS](/appstore/modules/genai/pgvector-setup/#aws-database-create) section. This is convenient, since PostgreSQL databases in Amazon RDS by default have the required pgvector extension available.{{% /alert %}} |
| 72 | +{{% alert color="info" %}}If you have access to an Amazon Web Services (AWS) account or Microsoft Azure account, Mendix recommends you use a setup described in the [Creating a PostgreSQL Database with Amazon RDS](/appstore/modules/genai/pgvector-setup/#aws-database-create) or [Managing a PostgreSQL Database with Microsoft Azure](/appstore/modules/genai/pgvector-setup/#azure-database) section. This is convenient, since these PostgreSQL databases in the cloud have the required pgvector extension available by default.{{% /alert %}} |
45 | 73 |
|
46 | 74 | ### Steps {#steps} |
47 | 75 |
|
@@ -88,8 +116,12 @@ If you would like to build your own RAG setup, feel free to learn from the GenAI |
88 | 116 | * Find top-k nearest neighbors (select query; typically using cosine distance/similarity optimization as recommended by OpenAI). |
89 | 117 |
|
90 | 118 | * Remove individual records (delete) or tables (drop table). |
| 119 | + |
| 120 | +* Similarity search is only guaranteed to work if the embeddings model chosen for the retrieval step is the same as the model used at the time of population: different models use different algorithms to generate vectors and might even produce vectors of different dimensionalities, making cosine distance calculation impossible. |
| 121 | + |
| 122 | +* How you construct the input string affects similarity search results. In the similarity search example for **tickets** in the showcase application, the input string at the time of insertion is a concatenation of multiple attributes of each ticket record in the Mendix database. However, in the search step, the user's input—possibly just a brief description—is used to find similar tickets. While this discrepancy may lower overall similarity, the most relevant records will still appear at the top. |
91 | 123 |
|
92 | | -{{% alert color="info" %}}Example queries in the form of SQL statements are available for inspiration in the source code of the [PgVector Knowledge Base module](/appstore/modules/genai/pgvector/) which comes automatically with GenAI Showcase App.{{% /alert %}} |
| 124 | +{{% alert color="info" %}}Reusable queries in the form of SQL statements are available in the source code of the [PgVector Knowledge Base module](/appstore/modules/genai/pgvector/) which comes automatically with GenAI Showcase App.{{% /alert %}} |
93 | 125 |
|
94 | 126 | ## Read More {#read-more} |
95 | 127 |
|
|
0 commit comments