Skip to content

Commit 902bdb0

Browse files
bsboddenbsbodden
committed
Adds self-contained docker environment for java-recipes
1 parent 41b96fa commit 902bdb0

File tree

11 files changed

+946
-0
lines changed

11 files changed

+946
-0
lines changed

‎.github/ignore-notebooks.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@
55
05_nvidia_ai_rag_redis
66
01_routing_optimization
77
02_semantic_cache_optimization
8+
spring_ai_redis_rag.ipynb

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,3 +220,4 @@ pip-selfcheck.json
220220
libs/redis/docs/.Trash*
221221
.python-version
222222
.idea/*
223+
java-recipes/.*

‎java-recipes/RAG/spring_ai_redis_rag.ipynb‎

Lines changed: 466 additions & 0 deletions
Large diffs are not rendered by default.

‎java-recipes/README.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
<div align="center">
2+
<div><img src="../assets/redis-logo.svg" style="width: 130px"> </div>
3+
<h1>Redis AI Java Resources</h1>
4+
<div align="center">
5+
6+
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
7+
![Java](https://img.shields.io/badge/Java-21-orange)
8+
![Spring AI](https://img.shields.io/badge/Spring%20AI-1.0.0--M6-green)
9+
10+
</div>
11+
<div>
12+
✨ Java-based code examples, notebooks, and resources for using Redis in AI and ML applications. ✨
13+
</div>
14+
15+
<div></div>
16+
<br>
17+
18+
[**Setup**](#setup) | [**Running the Project**](#running-the-project) | [**Notebooks**](#notebooks) | [**Project Structure**](#project-structure) | [**Implementation Details**](#implementation-details)
19+
20+
</div>
21+
<br>
22+
23+
## Setup
24+
25+
This project uses Docker Compose to set up a complete environment for running Java-based AI applications with Redis. The environment includes:
26+
27+
- A Jupyter Notebook server with Java kernel support
28+
- Redis Stack (includes Redis and RedisInsight)
29+
- Pre-installed dependencies for AI/ML workloads
30+
31+
### Prerequisites
32+
33+
- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/)
34+
- OpenAI API key (for notebooks that use OpenAI services)
35+
36+
### Environment Configuration
37+
38+
1. Create a `.env` file in the project root with your OpenAI API key:
39+
40+
```
41+
OPENAI_API_KEY=your_openai_api_key_here
42+
```
43+
44+
## Running the Project
45+
46+
1. Clone the repository (if you haven't already):
47+
48+
```bash
49+
git clone https://github.com/redis-developer/redis-ai-resources.git
50+
cd redis-ai-resources/java-resources
51+
```
52+
53+
2. Start the Docker containers:
54+
55+
```bash
56+
docker-compose up -d
57+
```
58+
59+
3. Access the Jupyter environment:
60+
- Open your browser and navigate to [http://localhost:8888](http://localhost:8888)
61+
- The token is usually shown in the docker-compose logs. You can view them with:
62+
63+
```bash
64+
docker-compose logs jupyter
65+
```
66+
67+
4. Access RedisInsight:
68+
- Open your browser and navigate to [http://localhost:8001](http://localhost:8001)
69+
- Connect to Redis using the following details:
70+
- Host: redis-java
71+
- Port: 6379
72+
- No password (unless configured)
73+
74+
5. When finished, stop the containers:
75+
76+
```bash
77+
docker-compose down
78+
```
79+
80+
## Notebooks
81+
82+
| Notebook | Description |
83+
| --- | --- |
84+
| [RAG/spring_ai_redis_rag.ipynb](./RAG/spring_ai_redis_rag.ipynb) | Demonstrates building a RAG-based beer recommendation chatbot using Spring AI and Redis as the vector store |
85+
86+
## Project Structure
87+
88+
```
89+
java-resources/
90+
├── .env # Environment variables (create this)
91+
├── docker-compose.yml # Docker Compose configuration
92+
├── jupyter/ # Jupyter configuration files
93+
│ ├── Dockerfile # Dockerfile for Jupyter with Java kernel
94+
│ ├── environment.yml # Conda environment specification
95+
│ ├── install.py # JJava kernel installation script
96+
│ ├── kernel.json # Kernel specification
97+
│ └── java/ # Java dependencies and configuration
98+
│ └── pom.xml # Maven project file with dependencies
99+
└── resources/ # Data files for notebooks
100+
└── beers.json.gz # Compressed beer dataset
101+
```
102+
103+
## Implementation Details
104+
105+
### Java Jupyter Kernel
106+
107+
The project uses [JJava](https://github.com/dflib/jjava), a Jupyter kernel for Java based on JShell. This allows for interactive Java development in Jupyter notebooks.
108+
109+
Key components:
110+
111+
- Java 21 for modern Java features
112+
- Maven for dependency management
113+
- JJava kernel for Jupyter integration
114+
115+
### Spring AI Integration
116+
117+
The Spring AI notebooks showcase how to use Spring's AI capabilities with Redis:
118+
119+
- **Spring AI**: Framework for building AI-powered applications
120+
- **Redis Vector Store**: Used for storing and querying vector embeddings
121+
- **Transformer Models**: For generating embeddings locally
122+
- **RAG Pattern**: Demonstrates the Retrieval Augmented Generation pattern
123+
124+
### Docker Configuration
125+
126+
The Docker setup includes:
127+
128+
1. **Jupyter Container**:
129+
- Based on minimal Jupyter notebook image
130+
- Adds Java 21, Maven, and the JJava kernel
131+
- Includes Python environment with PyTorch and other ML libraries
132+
133+
2. **Redis Container**:
134+
- Uses Redis Stack image with Vector Search capabilities
135+
- Persists data using Docker volumes
136+
- Exposes Redis on port 6379 and RedisInsight on port 8001
137+
138+
## Example Applications
139+
140+
### Beer Recommendation Chatbot
141+
142+
The `spring-ai-rag.ipynb` notebook demonstrates:
143+
144+
- Loading and embedding beer data into Redis Vector Store
145+
- Using local transformer models for generating embeddings
146+
- Connecting to OpenAI for LLM capabilities
147+
- Building a RAG pipeline to answer beer-related queries
148+
- Semantic search over beer properties and descriptions

‎java-recipes/docker-compose.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: redis-ai-java
2+
services:
3+
jupyter:
4+
build:
5+
context: .
6+
dockerfile: ./jupyter/Dockerfile
7+
ports:
8+
- "8888:8888"
9+
environment:
10+
- JUPYTER_ENABLE_LAB=yes
11+
env_file:
12+
- .env
13+
volumes:
14+
- ./:/home/jovyan/
15+
- ./resources:/home/jovyan/resources
16+
redis-java:
17+
image: redis/redis-stack:latest
18+
ports:
19+
- "6379:6379" # Redis database port
20+
- "8001:8001" # RedisInsight port
21+
volumes:
22+
- redis-data:/data # Persist Redis data
23+
24+
volumes:
25+
redis-data:

‎java-recipes/jupyter/Dockerfile‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
FROM quay.io/jupyter/minimal-notebook:latest
2+
3+
RUN mkdir /home/jovyan/resources
4+
5+
USER root
6+
WORKDIR /home/jovyan
7+
8+
# Install dependencies: Java 21 and Maven
9+
RUN apt-get update && apt-get install -y openjdk-21-jdk maven
10+
11+
# Copy the pre-created Maven project and jjava-glue project
12+
COPY ./jupyter/java /home/jovyan/java
13+
COPY ./jupyter/install.py /home/jovyan/install.py
14+
15+
# Use Maven to download dependencies for JJava
16+
WORKDIR /home/jovyan/java
17+
18+
# Download the JJava jar directly
19+
RUN mvn dependency:get -Dartifact=org.dflib.jjava:jjava:1.0-M3 -Ddest=./ -Dtransitive=false
20+
RUN mv jjava-1.0-M3.jar jjava.jar
21+
22+
# Pre-download Spring AI Dependencies
23+
RUN mvn dependency:get -Dartifact=org.springframework.ai:spring-ai-openai:1.0.0-M6
24+
RUN mvn dependency:get -Dartifact=org.springframework.ai:spring-ai-transformers:1.0.0-M6
25+
RUN mvn dependency:get -Dartifact=org.springframework.ai:spring-ai-redis-store:1.0.0-M6
26+
# Pre-download Jedis
27+
RUN mvn dependency:get -Dartifact=redis.clients:jedis:5.2.0
28+
# Download all dependencies
29+
RUN mvn dependency:copy-dependencies -DoutputDirectory=./lib
30+
31+
# Create a list of dependencies for the classpath
32+
RUN find ./lib -name "*.jar" | tr '\n' ':' > classpath.txt
33+
# Add the jjava.jar to the classpath
34+
RUN echo -n "/home/jovyan/java/jjava.jar:" >> classpath.txt
35+
36+
# Install the kernel with classpath configuration
37+
WORKDIR /home/jovyan
38+
RUN python install.py --prefix /opt/conda/ --classpath $(cat /home/jovyan/java/classpath.txt)
39+
40+
# Pre-download Transformer Models
41+
RUN pip install transformers torch
42+
RUN mkdir -p /home/jovyan/.cache/huggingface/hub
43+
# Pre-download the specific model used in Spring AI Transformers
44+
RUN python -c "from transformers import AutoModel; AutoModel.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')"
45+
46+
# Clean up Maven artifacts but keep the jjava.jar and lib directory
47+
RUN rm -rf /home/jovyan/java/target /home/jovyan/java/.m2 /home/jovyan/java/pom.xml \
48+
/home/jovyan/java/classpath.txt \
49+
&& rm -f /home/jovyan/install.py
50+
51+
# Install conda packages from environment.yml
52+
COPY ./jupyter/environment.yml /tmp/
53+
RUN conda env update -f /tmp/environment.yml && \
54+
conda clean --all -f -y && \
55+
fix-permissions "${CONDA_DIR}" && \
56+
fix-permissions "/home/${NB_USER}"
57+
58+
WORKDIR /home/jovyan
59+
USER $NB_UID

‎java-recipes/jupyter/environment.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
name: base
2+
channels:
3+
- pytorch
4+
- conda-forge
5+
- defaults
6+
dependencies:
7+
- pytorch
8+
- torchtext
9+
- gensim

0 commit comments

Comments
 (0)