Add support for Elasticsearch vector store #234

JM-Lab · 2024-01-09T13:50:32Z

This PR adds supports for Elasticsearch vector search.

To achieve compatibility with both Elasticsearch versions 7 and 8, I implemented it using Elasticsearch's Java Low Level REST Client.

To-do:

Add ElasticsearchAiSearchFilterExpressionConverter
Add support for FilterExpression
Add support for various vector functions provided by Elasticsearch

JM-Lab · 2024-01-14T10:35:35Z

Just finished up the Elasticsearch Vector Store support.
Next, I'll be working on adding support for OpenSearch (https://opensearch.org/) vector store.
Additionally, breaking down tasks for separate PR:

Add autoconfiguration
Write README

Let me know your thoughts.

markpollack · 2024-01-23T18:55:05Z

This is a great contribution. I believe @tzolov has started to review. I'll put it into 0.8.0 as a stretch goal for now.

tzolov

@JM-Lab, thank you for the contribution.

I started with a quick code check and have few questions related to the chosen ES client library and consequently the custom vs library query classes/records.

tzolov · 2024-01-26T08:53:07Z

vector-stores/spring-ai-elasticsearch-store/pom.xml

+			<version>${parent.version}</version>
+		</dependency>
+
+		<dependency>


Shouldn't we be using the elasticsearch-java client instead?

<dependency> <groupId>co.elastic.clients</groupId> <artifactId>elasticsearch-java</artifactId> <version>xxx</version> </dependency>

I chose to use the Java Low Level REST Client (https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/8.12/java-rest-low.html) with minimal dependencies instead of the elasticsearch-java library's High Level Rest Client. This decision was based on the High Level Rest Client's heavy reliance on dependencies and its sensitivity to Elasticsearch server versions, requiring exact matching with versions 7, 8, and even minor releases. By implementing only the essential data classes for Vectorstore, I successfully tested compatibility with both version 7 and 8.

tzolov · 2024-01-26T08:55:04Z

...csearch-store/src/main/java/org/springframework/ai/vectorstore/ElasticsearchVectorStore.java

+    public static final String COSINE_SIMILARITY_FUNCTION =
+            "(cosineSimilarity(params.query_vector, 'embedding') + 1.0) / 2";
+
+    public record ElasticsearchBulkIndexId(


Do we need all those custom records? Wouldn't the co.elastic client already provide such?

I focused on simplicity by implementing only the necessary custom classes/records for Vectorstore, minimizing dependencies and Elasticsearch version sensitivity as mentioned earlier.

tzolov · 2024-01-26T08:56:03Z

...csearch-store/src/main/java/org/springframework/ai/vectorstore/ElasticsearchVectorStore.java

+            @JsonProperty("query")
+            Query query
+    ) {
+        public record Query(


For example the isn't this already covered by the co.elastic.clients.elasticsearch._types.query_dsl.Query ?

I focused on simplicity by implementing only the necessary custom classes/records for Vectorstore, minimizing dependencies and Elasticsearch version sensitivity as mentioned earlier.

tzolov · 2024-01-26T08:58:37Z

...csearch-store/src/main/java/org/springframework/ai/vectorstore/ElasticsearchVectorStore.java

+
+    private ElasticsearchScriptScoreQuery getElasticsearchSimilarityQuery(List<Double> embedding, int topK,
+            float similarityThreshold, Filter.Expression filterExpression) {
+        return new ElasticsearchScriptScoreQuery(topK, new Query(


Again wouldn't the co.elastic.clients.elasticsearch._types.query_dsl.ScriptScoreQuery help here in places of custom records?

I focused on simplicity by implementing only the necessary custom classes/records for Vectorstore, minimizing dependencies and Elasticsearch version sensitivity as mentioned earlier.

JM-Lab · 2024-02-03T03:39:19Z

If we use elasticsearch-java, we might need to either split Vectorstore for Elasticsearch 7 and 8, or let users add elasticsearch-java version matching their elasticsearch server version. What's your preference? I'd love to hear your thoughts on this.

markpollack · 2024-02-29T15:47:23Z

I think moving to use the latest version that focuses on elastic search 8 and to build upon the elasticsearch-java will be the best use of everyone's time. Sorry for the delay in responding, but we are still very interested in this contribution and appreciate your interest/help.

JM-Lab · 2024-03-02T13:14:32Z

@markpollack I understand, and I'll update to support the latest version, Elasticsearch 8, and commit soon.

…java library

JM-Lab · 2024-03-03T14:34:57Z

@markpollack @tzolov Review code with Elasticsearch 8 support using elasticsearch-java library as discussed.

…tability VertexAiGeminiAutoConfiguration creates its own FunctionCallbackContext instance with schema type set to SchemaType.OPEN_API_SCHEMA. This ensures that one other (non OPEN_API_SCHEMA) FucntionCallbackContext Bean overrds it.

- Make it use the new OpenAiAudioApi. - Remove trascription code from spring-ai-core. Too early to generalize. Move all related code under the spring-ai-openai project. - Fix missing licenses and javadocs. - Add 'Audio' prefix for Transcription classes and packages. - Add missing auto-configuraiotn and tests.

- Move the OpenAiAudioTranscriptionClient and OpenAiAudioTranscriptionOptions under the org.springframework.ai.openai package. - Rename the AudioTranscriptionRequest into AudioTranscriptionPrompt. - Minor imports cleaning.

* root project pom now sets <dependencyManagement>remove</dependencyManagement> * spring-ai-bom pom flatten plugin added with <dependencyManagement>keep</dependencyManagement>

* Add test code

Unrelated to this change, the Neo4j test version increased to be current.

tzolov · 2024-03-15T17:00:27Z

Hi @JM-Lab ,
Finally i had some time today to start reviewing the PR.
After rebasing it on top of upstream main there were some compilation issues i had to fix.
Also i had to add the project the parent POM and to the BOM pom.

After this the filter expression test passes but the IT fails for half of the tests.
Here is my review branch: https://github.com/tzolov/spring-ai/pull/new/gh-234-pr

Please have a look, hopefully you would be able to resolve those.

- tename /api/clients/ into /api/chat - move the the image from /api/clients to /api - fix the layout inside the chat and embeddings docs. Moving the runtime options and sample controllers at top level. - adjust all affected links.

- Add VectorSearchAggregation used to actually preform the search on a given collection with embeddings. - add MongoDBVectorStore - Add MongoDBVectorStoreIT. Integration test runs fine given... - You have a mongo atlas cluster to connect to (local or remote) - You have the search index "spring_ai_vector_search" setup correctly - Need to explore getting around this - Need to filter results using threshold - Add postfilter for threshold values - While a post filter is not ideal, it gets the job done. The mongo team seems to be working on having it availible as a prefilter option, in which this implementation can be updated to use later. - implement filtering threshold - fix a few sonar issues - formatting - use higher default num_candidates - use builder for configuration - add documentation and some refactor - use consistent property in integration test - finish implementing filter support - add documentation to filter converter - add vector search index auto creation - Add to BOM. - Fix version to 1.0.0-SN. - Move expresion converter from core to models/mongodb. - Fix style and license headers

See spring-projectsgh-242

- Implemented Bedrock Jurassic ChatClient - added documentation reference - implement auto-configuration and boot starter - Disable the BedrockAi21Jurassic2ChatClientIT.emojiPenaltyWhenTrueByDefaultApplyPenaltyTest() test as it fails when run in combination with the other tests.

`Reuse Container` is a Testcontainers experimental feature. It requires `testcontainers.reuse.enable=true` in `~/.testcontainers.properties` in order to take effect but in order to avoid surprises, this commit remove it. See https://java.testcontainers.org/features/reuse/

…java library

…b/spring-ai into elasticsearch-vector-store

…arent pom and Bom pom

…es the use of epoch milliseconds

JM-Lab · 2024-03-17T06:44:30Z

Hi @tzolov

I've reviewed the PR.
I cherry-picked your commit 'Rebase, fix compilation and style errors, add missing dependencies in parent pom and Bom pom' and resolved the issues with Elasticsearch8VectorStoreIT. Additionally, I renamed it to ElasticsearchVectorStoreIT.

Please review the changes. Looking forward to your feedback!

tzolov · 2024-03-18T21:37:36Z

Thanks @JM-Lab ,
75 commits and 656 files changed, doesn't seem right.
Can you please squash the commits and make sure that only the ElasticSearch related files a touched.
It will be even better if you copy the ElasticSearch related files into a fresh new PR on top of the upstream main.

JM-Lab · 2024-03-19T13:02:01Z

@tzolov

I've just created a new PR focusing solely on the ElasticSearch files, on the main branch. You can find it here: #471.

Going to close this current one. Thanks!

tzolov · 2024-03-19T14:46:27Z

Thanks @JM-Lab I will have a look

JM-Lab added 2 commits January 9, 2024 22:42

Add ElasticsearchAiSearchFilterExpressionConverter

7294f7d

Add support for Elasticsearch vector store

c069db8

tzolov self-assigned this Jan 13, 2024

tzolov added vector store enhancement New feature or request labels Jan 13, 2024

Add support for various vector functions provided by Elasticsearch

8ff8b87

JM-Lab marked this pull request as ready for review January 14, 2024 10:07

markpollack mentioned this pull request Jan 23, 2024

Elastic/OpenSearch DB vector #257

Closed

markpollack added this to the 0.8.0 milestone Jan 23, 2024

tzolov reviewed Jan 26, 2024

View reviewed changes

markpollack modified the milestones: 0.8.0, 0.9.0 Feb 5, 2024

JM-Lab added 2 commits March 3, 2024 23:30

Add support for Elasticsearch 8 vector store using the elasticsearch-…

5d41978

…java library

Merge branch 'spring-projects:main' into elasticsearch-vector-store

011dfcb

tzolov and others added 10 commits March 4, 2024 13:58

Update README.md

db383f8

Improve the OpenAi Audio code structure

8ae506f

- Move the OpenAiAudioTranscriptionClient and OpenAiAudioTranscriptionOptions under the org.springframework.ai.openai package. - Rename the AudioTranscriptionRequest into AudioTranscriptionPrompt. - Minor imports cleaning.

Upgrade Azure OpenAI from 1.0.0-beta.6 to 1.0.0-beta.7

c0c9dd2

Change flatten pom config

7cc2eab

* root project pom now sets <dependencyManagement>remove</dependencyManagement> * spring-ai-bom pom flatten plugin added with <dependencyManagement>keep</dependencyManagement>

Update to add latest gpt-3.5 turbo model number

7396597

Remove setters from options interface

3938cc8

* Add test code

rename directory spring-ai-stabilityai to stability-ai

f6c57f6

add license header plugin and update all java files

bcb559a

meistermeier and others added 8 commits March 15, 2024 12:26

Allow more customization for Neo4j store (id and constraint).

7c6bbef

Unrelated to this change, the Neo4j test version increased to be current.

fix code formatting

6638efe

modified ImageMessage.java line 45: mage > Image

0e8b696

fixing GH document upload action

a8142b7

Fix GH documenttation-upload.yml

4e3bba7

Improve the Deploy docs steps of Docs uplaod GH action

cf463c8

Final GH doc action fixes

c453529

Improve auto-confg condigiton on class to prevent undesired activation

9330187

tzolov and others added 14 commits March 16, 2024 10:53

Update Mistral AI function calling docs and layout

49a1cc6

Improve Antora documentation layout

9c19dc1

- tename /api/clients/ into /api/chat - move the the image from /api/clients to /api - fix the layout inside the chat and embeddings docs. Moving the runtime options and sample controllers at top level. - adjust all affected links.

Add PostgresMlAutoConfiguration to AutoConfiguration.imports

db43fe3

See spring-projectsgh-242

Add ElasticsearchAiSearchFilterExpressionConverter

c2b3659

Add support for Elasticsearch vector store

04b25ff

Add support for various vector functions provided by Elasticsearch

bdc413c

Add support for Elasticsearch 8 vector store using the elasticsearch-…

d0a185b

…java library

Merge branch 'elasticsearch-vector-store' of https://github.com/JM-La…

ba1728e

…b/spring-ai into elasticsearch-vector-store

Rebase, fix compilation and style errors add missing dependecies in p…

b38552c

…arent pom and Bom pom

Fix ElasticsearchVectorStoreIT FilterExpression with Date type requir…

a28f245

…es the use of epoch milliseconds

Rename Elasticsearch8VectorStoreIT to ElasticsearchVectorStoreIT

5cb6d92

JM-Lab closed this Mar 19, 2024

JM-Lab mentioned this pull request Mar 19, 2024

Add support for Elasticsearch vector store: Squashed #471

Closed

JM-Lab deleted the elasticsearch-vector-store branch March 22, 2024 06:07

Add support for Elasticsearch vector store #234

Add support for Elasticsearch vector store #234

Uh oh!

Conversation

JM-Lab commented Jan 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JM-Lab commented Jan 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markpollack commented Jan 23, 2024

Uh oh!

tzolov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JM-Lab commented Feb 3, 2024

Uh oh!

markpollack commented Feb 29, 2024

Uh oh!

JM-Lab commented Mar 2, 2024

Uh oh!

JM-Lab commented Mar 3, 2024

Uh oh!

tzolov commented Mar 15, 2024

Uh oh!

JM-Lab commented Mar 17, 2024

Uh oh!

tzolov commented Mar 18, 2024

Uh oh!

JM-Lab commented Mar 19, 2024

Uh oh!

tzolov commented Mar 19, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

18 participants

JM-Lab commented Jan 9, 2024 •

edited

Loading

JM-Lab commented Jan 14, 2024 •

edited

Loading