Skip to content

Commit 3305aec

Browse files
tylerhutchersonMichael Yuanjoeywhelan
authored
Additional client examples (#15)
* add repo structure * Add nodejs examples (#11) * Add Java vector database examples (#14) * Update config.properties * Update README.md * Delete vector-database/python directory * Update README.md Co-authored-by: Michael Yuan <[email protected]> Co-authored-by: Joey Whelan <[email protected]>
1 parent 71131cf commit 3305aec

32 files changed

+94716
-0
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.env
2+
node_modules/

‎vector-database/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Vector Database Examples
2+
3+
Below we provide Redis vector database dev examples by language and by feature.
4+
5+
- [Python]([python/](https://github.com/openai/openai-cookbook/tree/main/examples/vector_databases/redis)https://github.com/openai/openai-cookbook/tree/main/examples/vector_databases/redis) - From the Official OpenAI cookbook
6+
- [NodeJS](nodejs/)
7+
- [Java](java/)

‎vector-database/java/LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2023 Michael Yuan
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎vector-database/java/README.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# Redis VSS OpenAI Examples in Java
2+
3+
## Contents
4+
1. [Summary](#summary)
5+
2. [Features](#features)
6+
3. [Prerequisites](#prerequisites)
7+
4. [Installation](#installation)
8+
5. [Usage](#usage)
9+
6. [Execution](#execution)
10+
11+
## Summary <a name="summary"></a>
12+
This provides a series of Java code examples of how to use Redis VSS with vector embeddings generated with OpenAI.
13+
14+
## Features <a name="features"></a>
15+
- Java source code for implementing Redis VSS on JSON documents using an ecommerce dataset of products and obtaining vector embeddings from OpenAI.
16+
- Java source code for implementing Redis VSS on Redis Hashes using a dataset with wikipedia articles with OpenAI vector embeddings.
17+
- Docker compose file to start up a Redis Stack instance.
18+
19+
## Prerequisites <a name="prerequisites"></a>
20+
- Docker
21+
- Java JDK
22+
- [OpenAI key](https://platform.openai.com)
23+
24+
## Installation <a name="installation"></a>
25+
1. Clone this repo.
26+
2. CD to the java directory
27+
3. Export an environment variable with your OpenAI API key: ```export OPENAI_API_KEY="<YOUR_KEY>"```
28+
4. Start up Redis Stack: docker compose up -d
29+
5. Using a JDK or Java IDE build the maven project and run the examples
30+
31+
## Execution <a name="execution"></a>
32+
### Redis Client Connection
33+
```java
34+
client = new JedisPooled(prop.getProperty("redis.host"),
35+
Integer.parseInt(prop.getProperty("redis.port")));
36+
```
37+
### OpenAI Client Connection
38+
```java
39+
service = new OpenAiService(token);
40+
```
41+
### Flat Index Build
42+
```java
43+
// Define index schema
44+
Schema schema = new Schema().addNumericField("id")
45+
.addTextField("title", 3.0).as("title")
46+
.addTextField("url", 1.0).as("url")
47+
.addTextField("text", 2.0).as("text")
48+
.addVectorField("title_vector", Schema.VectorField.VectorAlgo.FLAT, attr).as("title_vector")
49+
.addVectorField("content_vector", Schema.VectorField.VectorAlgo.FLAT, attr).as("content_vector");
50+
IndexDefinition rule = new IndexDefinition(IndexDefinition.Type.HASH)
51+
.setPrefixes(new String[] { "wiki:" });
52+
client.ftCreate(INDEX_NAME, IndexOptions.defaultOptions().setDefinition(rule), schema);
53+
```
54+
### HNSW Index Build
55+
```java
56+
// Define index schema
57+
Schema schema = new Schema().addNumericField("id")
58+
.addTextField("title", 3.0).as("title")
59+
.addTextField("url", 1.0).as("url")
60+
.addTextField("text", 2.0).as("text")
61+
.addVectorField("title_vector", Schema.VectorField.VectorAlgo.HNSW, attr).as("title_vector")
62+
.addVectorField("content_vector", Schema.VectorField.VectorAlgo.HNSW, attr).as("content_vector");
63+
IndexDefinition rule = new IndexDefinition(IndexDefinition.Type.HASH)
64+
.setPrefixes(new String[] { "wiki:" });
65+
client.ftCreate(INDEX_NAME_HNSW, IndexOptions.defaultOptions().setDefinition(rule), schema);
66+
```
67+
### Redis Hash Data Load
68+
```java
69+
Map<byte[], byte[]> map = new HashMap<>();
70+
map.put("id".getBytes(), record[0].getBytes());
71+
map.put("url".getBytes(), record[1].getBytes());
72+
map.put("title".getBytes(), record[2].getBytes());
73+
map.put("text".getBytes(), record[3].getBytes());
74+
map.put("title_vector".getBytes(), doubleToByte(title_vector));
75+
map.put("content_vector".getBytes(), doubleToByte(content_vector));
76+
map.put("vector_id".getBytes(), record[6].getBytes());
77+
client.hset(key.getBytes(), map);
78+
```
79+
### Redis JSON Data Load
80+
```java
81+
Product product = productList.get(i);
82+
product.addVector(embeddings.get(i).getEmbedding().stream().mapToDouble(Double::doubleValue)
83+
.toArray());
84+
client.jsonSet("product:" + product.id, gson.toJson(product));
85+
```
86+
### VSS query: 'modern art in Europe' in 'title_vector'
87+
```text
88+
1. Museum of Modern Art (Score: 0.8751771)
89+
2. Western Europe (Score: 0.8674411)
90+
3. Renaissance art (Score: 0.86415625)
91+
4. Pop art (Score: 0.8603469)
92+
5. Northern Europe (Score: 0.85465807)
93+
6. Hellenistic art (Score: 0.8527923)
94+
7. Modernist literature (Score: 0.84703135)
95+
8. Art film (Score: 0.84327316)
96+
9. Central Europe (Score: 0.84258366)
97+
10. European (Score: 0.84141064)
98+
```
99+
### VSS query: 'Famous battles in Scottish history' in 'content_vector'
100+
```text
101+
1. Battle of Bannockburn (Score: 0.86933625)
102+
2. Wars of Scottish Independence (Score: 0.8614707)
103+
3. 1651 (Score: 0.85258836)
104+
4. First War of Scottish Independence (Score: 0.84962213)
105+
5. Robert I of Scotland (Score: 0.84621406)
106+
6. 841 (Score: 0.84399074)
107+
7. 1716 (Score: 0.84390485)
108+
8. 1314 (Score: 0.83721495)
109+
9. 1263 (Score: 0.8364166)
110+
10. William Wallace (Score: 0.83534056)
111+
```
112+
### VSS query: 'man blue jeans' in 'productVector'
113+
```text
114+
1. John Players Men Blue Jeans (Score: 0.79446274)
115+
2. Lee Men Tino Blue Jeans (Score: 0.7797863)
116+
3. Lee Men Blue Chicago Fit Jeans (Score: 0.77107173)
117+
4. Lee Men Blue Chicago Fit Jeans (Score: 0.7710503)
118+
5. Peter England Men Party Blue Jeans (Score: 0.7699358)
119+
6. Locomotive Men Washed Blue Jeans (Score: 0.74747217)
120+
7. Locomotive Men Washed Blue Jeans (Score: 0.74747217)
121+
8. French Connection Men Blue Jeans (Score: 0.7463001)
122+
9. Palm Tree Kids Boy Washed Blue Jeans (Score: 0.7440362)
123+
10. Lee Men Elvira Rinse Blue Chicago Fit Jeans (Score: 0.7366651)
124+
```
125+
### VSS hybrid query: 'man blue jeans' in 'productVector' with hybrid filters: @productDisplayName:"slim fit"
126+
```text
127+
1. Lee Rinse Navy Blue Slim Fit Jeans (Score: 0.71524847)
128+
2. Basics Men Blue Slim Fit Checked Shirt (Score: 0.71524143)
129+
3. Basics Men Blue Slim Fit Checked Shirt (Score: 0.71524143)
130+
4. Tokyo Talkies Women Navy Slim Fit Jeans (Score: 0.6794758)
131+
5. Basics Men Navy Slim Fit Checked Shirt (Score: 0.6708832)
132+
6. Basics Men Red Slim Fit Checked Shirt (Score: 0.6275975)
133+
7. Basics Men White Slim Fit Striped Shirt (Score: 0.622632)
134+
8. ADIDAS Men's Slim Fit White T-shirt (Score: 0.5857945)
135+
```
136+
### VSS hybrid query: 'man blue jeans' in 'productVector' with hybrid filters: (@year:[2011 2012] @season:{Summer})
137+
```text
138+
1. John Players Men Blue Jeans (Score: 0.79437006)
139+
2. Peter England Men Party Blue Jeans (Score: 0.7699023)
140+
3. French Connection Men Blue Jeans (Score: 0.746287)
141+
4. Denizen Women Blue Jeans (Score: 0.7350321)
142+
5. Do U Speak Green Men Blue Shorts (Score: 0.7293731)
143+
6. John Players Men Check Blue Shirt (Score: 0.7255274)
144+
7. Jealous 21 Women Washed Blue Jeans (Score: 0.7209517)
145+
8. Jealous 21 Women Washed Blue Jeans (Score: 0.7209517)
146+
9. Lee Men Solid Blue Shirts (Score: 0.7166283)
147+
10. Gini and Jony Boys Check Blue Shirt (Score: 0.7127073)
148+
```

‎vector-database/java/docker-compose.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
version: '3.7'
2+
services:
3+
4+
vector-db:
5+
image: redis/redis-stack:latest
6+
ports:
7+
- 6379:6379
8+
- 8001:8001
9+
environment:
10+
- REDISEARCH_ARGS=CONCURRENT_WRITE_MODE
11+
volumes:
12+
- vector-db:/var/lib/redis
13+
- ./redis.conf:/usr/local/etc/redis/redis.conf
14+
healthcheck:
15+
test: ["CMD", "redis-cli", "-h", "localhost", "-p", "6379", "ping"]
16+
interval: 2s
17+
timeout: 1m30s
18+
retries: 5
19+
start_period: 5s
20+
21+
volumes:
22+
vector-db:

0 commit comments

Comments
 (0)