Skip to content

Commit 82f6d29

Browse files
edeandreaedeandrea
committed
Reuse embeddings
Closes #663
1 parent 3c9bf71 commit 82f6d29

File tree

17 files changed

+570
-14
lines changed

17 files changed

+570
-14
lines changed

‎docs/modules/ROOT/assets/images/easy-rag-reuse-embeddings.png‎

68 KB
Loading

‎docs/modules/ROOT/pages/easy-rag.adoc‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,26 @@ NOTE: If you add two or more artifacts that provide embedding models,
4141
Quarkus will ask you to choose one of them using the
4242
`quarkus.langchain4j.embedding-model.provider` property.
4343

44+
== Reusing embeddings
45+
46+
You might find that when doing local development that computing embeddings takes some time. This pain can often be felt in dev mode when it may take several minutes between restarts.
47+
48+
That's where reusable ingestions come in! If you don't configure a persistent embedding store and set
49+
50+
[source,properties,subs=attributes+]
51+
----
52+
quarkus.langchain4j.easy-rag.reuse-embeddings.enabled=true
53+
----
54+
55+
then the `easy-rag` extension will detect the local embeddings. If they exist then they are loaded into the embedding store. If they don't exist, they are computed as normal and then written out to the file `easy-rag-embeddings.json` in the current directory.
56+
57+
NOTE: You can customize the embeddings file by setting the `quarkus.langchain4j.easy-rag.reuse-embeddings.file` property.
58+
59+
See this diagram which describes the flow:
60+
61+
.Reusable ingestion flow
62+
image::easy-rag-reuse-embeddings.png[align="center"]
63+
4464
== Getting started with a ready-to-use example
4565

4666
To see Easy RAG in action, use the project `samples/chatbot-easy-rag` in the
@@ -83,4 +103,10 @@ meaning all files recursively.
83103
For finer-grained control of the Apache Tika parsers (for example, to turn
84104
off OCR capabilities), you can use a regular XML config file recognized by
85105
Tika (see https://tika.apache.org/2.9.2/configuring.html[Tika
86-
documentation]), and specify `-Dtika.config` to point at the file.
106+
documentation]), and specify `-Dtika.config` to point at the file.
107+
108+
== Configuration
109+
110+
Several configuration properties are available:
111+
112+
include::includes/quarkus-langchain4j-easy-rag.adoc[leveloffset=+1,opts=optional]

‎docs/modules/ROOT/pages/includes/quarkus-langchain4j-easy-rag.adoc‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
2+
:summaryTableId: quarkus-langchain4j-easy-rag
3+
[.configuration-legend]
4+
icon:lock[title=Fixed at build time] Configuration property fixed at build time - All other configuration properties are overridable at runtime
5+
[.configuration-reference.searchable, cols="80,.^10,.^10"]
6+
|===
7+
8+
h|[[quarkus-langchain4j-easy-rag_configuration]]link:#quarkus-langchain4j-easy-rag_configuration[Configuration property]
9+
10+
h|Type
11+
h|Default
12+
13+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-path]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-path[quarkus.langchain4j.easy-rag.path]`
14+
15+
16+
[.description]
17+
--
18+
Path to the directory containing the documents to be ingested.
19+
20+
ifdef::add-copy-button-to-env-var[]
21+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_PATH+++[]
22+
endif::add-copy-button-to-env-var[]
23+
ifndef::add-copy-button-to-env-var[]
24+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_PATH+++`
25+
endif::add-copy-button-to-env-var[]
26+
--|string
27+
|required icon:exclamation-circle[title=Configuration property is required]
28+
29+
30+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-path-matcher]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-path-matcher[quarkus.langchain4j.easy-rag.path-matcher]`
31+
32+
33+
[.description]
34+
--
35+
Matcher used for filtering which files from the directory should be ingested. This uses the `java.nio.file.FileSystem` path matcher syntax. Example: `glob:++**++.txt` to recursively match all files with the `.txt` extension. The default is `glob:++**++`, recursively matching all files.
36+
37+
ifdef::add-copy-button-to-env-var[]
38+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_PATH_MATCHER+++[]
39+
endif::add-copy-button-to-env-var[]
40+
ifndef::add-copy-button-to-env-var[]
41+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_PATH_MATCHER+++`
42+
endif::add-copy-button-to-env-var[]
43+
--|string
44+
|`glob:**`
45+
46+
47+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-recursive]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-recursive[quarkus.langchain4j.easy-rag.recursive]`
48+
49+
50+
[.description]
51+
--
52+
Whether to recursively ingest documents from subdirectories.
53+
54+
ifdef::add-copy-button-to-env-var[]
55+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_RECURSIVE+++[]
56+
endif::add-copy-button-to-env-var[]
57+
ifndef::add-copy-button-to-env-var[]
58+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_RECURSIVE+++`
59+
endif::add-copy-button-to-env-var[]
60+
--|boolean
61+
|`true`
62+
63+
64+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-max-segment-size]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-max-segment-size[quarkus.langchain4j.easy-rag.max-segment-size]`
65+
66+
67+
[.description]
68+
--
69+
Maximum segment size when splitting documents, in tokens.
70+
71+
ifdef::add-copy-button-to-env-var[]
72+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_MAX_SEGMENT_SIZE+++[]
73+
endif::add-copy-button-to-env-var[]
74+
ifndef::add-copy-button-to-env-var[]
75+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_MAX_SEGMENT_SIZE+++`
76+
endif::add-copy-button-to-env-var[]
77+
--|int
78+
|`300`
79+
80+
81+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-max-overlap-size]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-max-overlap-size[quarkus.langchain4j.easy-rag.max-overlap-size]`
82+
83+
84+
[.description]
85+
--
86+
Maximum overlap (in tokens) when splitting documents.
87+
88+
ifdef::add-copy-button-to-env-var[]
89+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_MAX_OVERLAP_SIZE+++[]
90+
endif::add-copy-button-to-env-var[]
91+
ifndef::add-copy-button-to-env-var[]
92+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_MAX_OVERLAP_SIZE+++`
93+
endif::add-copy-button-to-env-var[]
94+
--|int
95+
|`30`
96+
97+
98+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-max-results]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-max-results[quarkus.langchain4j.easy-rag.max-results]`
99+
100+
101+
[.description]
102+
--
103+
Maximum number of results to return when querying the retrieval augmentor.
104+
105+
ifdef::add-copy-button-to-env-var[]
106+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_MAX_RESULTS+++[]
107+
endif::add-copy-button-to-env-var[]
108+
ifndef::add-copy-button-to-env-var[]
109+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_MAX_RESULTS+++`
110+
endif::add-copy-button-to-env-var[]
111+
--|int
112+
|`5`
113+
114+
115+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-ingestion-strategy]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-ingestion-strategy[quarkus.langchain4j.easy-rag.ingestion-strategy]`
116+
117+
118+
[.description]
119+
--
120+
The strategy to decide whether document ingestion into the store should happen at startup or not. The default is ON. Changing to OFF generally only makes sense if running against a persistent embedding store that was already populated.
121+
122+
ifdef::add-copy-button-to-env-var[]
123+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_INGESTION_STRATEGY+++[]
124+
endif::add-copy-button-to-env-var[]
125+
ifndef::add-copy-button-to-env-var[]
126+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_INGESTION_STRATEGY+++`
127+
endif::add-copy-button-to-env-var[]
128+
-- a|
129+
`on`, `off`
130+
|`on`
131+
132+
133+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-reuse-embeddings-enabled]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-reuse-embeddings-enabled[quarkus.langchain4j.easy-rag.reuse-embeddings.enabled]`
134+
135+
136+
[.description]
137+
--
138+
Whether or not to reuse embeddings. Defaults to `false`.
139+
140+
ifdef::add-copy-button-to-env-var[]
141+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_REUSE_EMBEDDINGS_ENABLED+++[]
142+
endif::add-copy-button-to-env-var[]
143+
ifndef::add-copy-button-to-env-var[]
144+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_REUSE_EMBEDDINGS_ENABLED+++`
145+
endif::add-copy-button-to-env-var[]
146+
--|boolean
147+
|`false`
148+
149+
150+
a| [[quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-reuse-embeddings-file]]`link:#quarkus-langchain4j-easy-rag_quarkus-langchain4j-easy-rag-reuse-embeddings-file[quarkus.langchain4j.easy-rag.reuse-embeddings.file]`
151+
152+
153+
[.description]
154+
--
155+
The file path to load/save embeddings, assuming `quarkus.langchain4j.easy-rag.reuse-embeddings.enabled == true`.
156+
157+
Defaults to `easy-rag-embeddings.json` in the current directory.
158+
159+
ifdef::add-copy-button-to-env-var[]
160+
Environment variable: env_var_with_copy_button:+++QUARKUS_LANGCHAIN4J_EASY_RAG_REUSE_EMBEDDINGS_FILE+++[]
161+
endif::add-copy-button-to-env-var[]
162+
ifndef::add-copy-button-to-env-var[]
163+
Environment variable: `+++QUARKUS_LANGCHAIN4J_EASY_RAG_REUSE_EMBEDDINGS_FILE+++`
164+
endif::add-copy-button-to-env-var[]
165+
--|string
166+
|`easy-rag-embeddings.json`
167+
168+
|===

‎docs/pom.xml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,11 @@
7676
<artifactId>quarkus-langchain4j-watsonx</artifactId>
7777
<version>${project.version}</version>
7878
</dependency>
79+
<dependency>
80+
<groupId>io.quarkiverse.langchain4j</groupId>
81+
<artifactId>quarkus-langchain4j-easy-rag</artifactId>
82+
<version>${project.version}</version>
83+
</dependency>
7984
<dependency>
8085
<groupId>io.quarkiverse.antora</groupId>
8186
<artifactId>quarkus-antora</artifactId>
@@ -100,6 +105,19 @@
100105
<artifactId>quarkus-langchain4j-anthropic-deployment</artifactId>
101106
<version>${project.version}</version>
102107
</dependency>
108+
<dependency>
109+
<groupId>io.quarkiverse.langchain4j</groupId>
110+
<artifactId>quarkus-langchain4j-easy-rag-deployment</artifactId>
111+
<version>${project.version}</version>
112+
<type>pom</type>
113+
<scope>test</scope>
114+
<exclusions>
115+
<exclusion>
116+
<groupId>*</groupId>
117+
<artifactId>*</artifactId>
118+
</exclusion>
119+
</exclusions>
120+
</dependency>
103121
<dependency>
104122
<groupId>io.quarkiverse.langchain4j</groupId>
105123
<artifactId>quarkus-langchain4j-openai-deployment</artifactId>
@@ -310,6 +328,7 @@
310328
<directory>${project.basedir}/../target/asciidoc/generated/config/</directory>
311329
<include>quarkus-langchain4j.adoc</include>
312330
<include>quarkus-langchain4j-anthropic.adoc</include>
331+
<include>quarkus-langchain4j-easy-rag.adoc</include>
313332
<include>quarkus-langchain4j-openai.adoc</include>
314333
<include>quarkus-langchain4j-huggingface.adoc</include>
315334
<include>quarkus-langchain4j-ollama.adoc</include>

‎docs/src/main/resources/application.properties‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ quarkus.langchain4j.pinecone.index-name=abc
1010
quarkus.langchain4j.pinecone.project-id=abc
1111
quarkus.langchain4j.pinecone.api-key=abc
1212
quarkus.langchain4j.redis.dimension=180
13+
quarkus.langchain4j.easy-rag.path=abc
14+
quarkus.langchain4j.easy-rag.ingestion-strategy=off
1315
quarkus.redis.hosts=redis://localhost:6379
1416

1517

‎rag/easy-rag/deployment/src/main/java/io/quarkiverse/langchain4j/deployment/EasyRagProcessor.java‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ public void createInMemoryEmbeddingStoreIfNoOtherExists(
5555
BuildProducer<SyntheticBeanBuildItem> beanProducer,
5656
List<EmbeddingStoreBuildItem> embeddingStores,
5757
EasyRagRecorder recorder,
58+
EasyRagConfig config,
5859
BuildProducer<InMemoryEmbeddingStoreBuildItem> inMemoryEmbeddingStoreBuildItemBuildProducer) {
5960
if (embeddingStores.isEmpty()) {
6061
beanProducer.produce(SyntheticBeanBuildItem
@@ -68,7 +69,7 @@ public void createInMemoryEmbeddingStoreIfNoOtherExists(
6869
.defaultBean()
6970
.unremovable()
7071
.scope(ApplicationScoped.class)
71-
.supplier(recorder.inMemoryEmbeddingStoreSupplier())
72+
.supplier(recorder.inMemoryEmbeddingStoreSupplier(config))
7273
.done());
7374
inMemoryEmbeddingStoreBuildItemBuildProducer.produce(new InMemoryEmbeddingStoreBuildItem());
7475
}

‎rag/easy-rag/deployment/src/test/java/io/quarkiverse/langchain4j/test/EasyRagNotRecursiveTest.java‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
package io.quarkiverse.langchain4j.test;
22

3+
import static org.assertj.core.api.Assertions.assertThat;
34
import static org.junit.jupiter.api.Assertions.assertEquals;
45
import static org.junit.jupiter.api.Assertions.assertTrue;
56

67
import java.util.List;
8+
import java.util.logging.LogRecord;
79

810
import jakarta.inject.Inject;
911

@@ -29,13 +31,24 @@ public class EasyRagNotRecursiveTest {
2931
.setArchiveProducer(() -> ShrinkWrap.create(JavaArchive.class)
3032
.addAsResource(new StringAsset("quarkus.langchain4j.easy-rag.path=src/test/resources/ragdocuments\n" +
3133
"quarkus.langchain4j.easy-rag.recursive=false\n"),
32-
"application.properties"));
34+
"application.properties"))
35+
.setLogRecordPredicate(record -> true)
36+
.assertLogRecords(EasyRagNotRecursiveTest::verifyLogRecords);
3337

3438
@Inject
3539
InMemoryEmbeddingStore<TextSegment> embeddingStore;
3640

3741
Embedding DUMMY_EMBEDDING = new Embedding(new float[384]);
3842

43+
private static void verifyLogRecords(List<LogRecord> logRecords) {
44+
assertThat(logRecords.stream().map(LogRecord::getMessage))
45+
.contains(
46+
"Ingesting documents from path: src/test/resources/ragdocuments, path matcher = glob:**, recursive = false")
47+
.contains("Ingested 1 files as 1 documents")
48+
.doesNotContain("Writing embeddings to %s")
49+
.doesNotContain("Reading embeddings from %s");
50+
}
51+
3952
@Test
4053
public void verifyOnlyTheRootDirectoryIsIngested() {
4154
List<EmbeddingMatch<TextSegment>> relevant = embeddingStore.findRelevant(DUMMY_EMBEDDING, 3);

‎rag/easy-rag/deployment/src/test/java/io/quarkiverse/langchain4j/test/EasyRagPathMatcherTest.java‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
package io.quarkiverse.langchain4j.test;
22

3+
import static org.assertj.core.api.Assertions.assertThat;
34
import static org.junit.jupiter.api.Assertions.*;
45

56
import java.util.List;
7+
import java.util.logging.LogRecord;
68

79
import jakarta.inject.Inject;
810

@@ -28,13 +30,24 @@ public class EasyRagPathMatcherTest {
2830
.setArchiveProducer(() -> ShrinkWrap.create(JavaArchive.class)
2931
.addAsResource(new StringAsset("quarkus.langchain4j.easy-rag.path=src/test/resources/ragdocuments\n" +
3032
"quarkus.langchain4j.easy-rag.path-matcher=glob:*.pdf\n"),
31-
"application.properties"));
33+
"application.properties"))
34+
.setLogRecordPredicate(record -> true)
35+
.assertLogRecords(EasyRagPathMatcherTest::verifyLogRecords);
3236

3337
@Inject
3438
InMemoryEmbeddingStore<TextSegment> embeddingStore;
3539

3640
Embedding DUMMY_EMBEDDING = new Embedding(new float[384]);
3741

42+
private static void verifyLogRecords(List<LogRecord> logRecords) {
43+
assertThat(logRecords.stream().map(LogRecord::getMessage))
44+
.contains(
45+
"Ingesting documents from path: src/test/resources/ragdocuments, path matcher = glob:*.pdf, recursive = true")
46+
.contains("Ingested 1 files as 1 documents")
47+
.doesNotContain("Writing embeddings to %s")
48+
.doesNotContain("Reading embeddings from %s");
49+
}
50+
3851
@Test
3952
public void verifyPathMatchingOnlyPdf() {
4053
List<EmbeddingMatch<TextSegment>> relevant = embeddingStore.findRelevant(DUMMY_EMBEDDING, 3);

0 commit comments

Comments
 (0)