diff --git a/java-recipes/README.md b/java-recipes/README.md index 0d7147d0..558c9dac 100644 --- a/java-recipes/README.md +++ b/java-recipes/README.md @@ -1,5 +1,5 @@
-
+

Redis AI Java Resources

@@ -15,127 +15,25 @@

-[**Setup**](#setup) | [**Running the Project**](#running-the-project) | [**Notebooks**](#notebooks) | [**Project Structure**](#project-structure) | [**Implementation Details**](#implementation-details) +[**Notebooks**](#notebooks) | [**Applications**](#applications) | [**Example Applications**](#example-applications)

-## Setup - -This project uses Docker Compose to set up a complete environment for running Java-based AI applications with Redis. The environment includes: - -- A Jupyter Notebook server with Java kernel support -- Redis Stack (includes Redis and RedisInsight) -- Pre-installed dependencies for AI/ML workloads - -### Prerequisites - -- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) -- OpenAI API key (for notebooks that use OpenAI services) - -### Environment Configuration - -1. Create a `.env` file in the project root with your OpenAI API key: - -```bash -OPENAI_API_KEY=your_openai_api_key_here -``` - -## Running the Project - -1. Clone the repository (if you haven't already): - - ```bash - git clone https://github.com/redis-developer/redis-ai-resources.git - cd redis-ai-resources/java-resources - ``` - -2. Start the Docker containers: - - ```bash - docker-compose up -d - ``` - -3. Access the Jupyter environment: - - Open your browser and navigate to [http://localhost:8888](http://localhost:8888) - - The token is usually shown in the docker-compose logs. You can view them with: - - ```bash - docker-compose logs jupyter - ``` - -4. Access RedisInsight: - - Open your browser and navigate to [http://localhost:8001](http://localhost:8001) - - Connect to Redis using the following details: - - Host: redis-java - - Port: 6379 - - No password (unless configured) - -5. When finished, stop the containers: - - ```bash - docker-compose down - ``` - ## Notebooks -| Notebook | Description | -| --- | --- | -| [RAG/spring_ai_redis_rag.ipynb](./RAG/spring_ai_redis_rag.ipynb) | Demonstrates building a RAG-based beer recommendation chatbot using Spring AI and Redis as the vector store | - -## Project Structure - -```bash -java-recipes/ -├── .env # Environment variables (create this) -├── docker-compose.yml # Docker Compose configuration -├── jupyter/ # Jupyter configuration files -│ ├── Dockerfile # Dockerfile for Jupyter with Java kernel -│ ├── environment.yml # Conda environment specification -│ ├── install.py # JJava kernel installation script -│ ├── kernel.json # Kernel specification -│ └── java/ # Java dependencies and configuration -│ └── pom.xml # Maven project file with dependencies -└── resources/ # Data files for notebooks - └── beers.json.gz # Compressed beer dataset -``` - -## Implementation Details - -### Java Jupyter Kernel - -The project uses [JJava](https://github.com/dflib/jjava), a Jupyter kernel for Java based on JShell. This allows for interactive Java development in Jupyter notebooks. - -Key components: +| Notebook | Description | +|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------| +| [notebooks/RAG/spring_ai_redis_rag.ipynb](./notebooks/RAG/spring_ai_redis_rag.ipynb) | Demonstrates building a RAG-ba sed beer recommendation chatbot using Spring AI and Redis as the vector store | -- Java 21 for modern Java features -- Maven for dependency management -- JJava kernel for Jupyter integration +## Applications -### Spring AI Integration +| Application | Description | +|-----------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------| +| [applications/vector-similarity-search/spring_boot](./applications/vector-similarity-search/spring_boot_redis_om_spring.md) | Demonstrates building a vector similarity search application using Spring Boot and Redis OM Spring | -The Spring AI notebooks showcase how to use Spring's AI capabilities with Redis: -- **Spring AI**: Framework for building AI-powered applications -- **Redis Vector Store**: Used for storing and querying vector embeddings -- **Transformer Models**: For generating embeddings locally -- **RAG Pattern**: Demonstrates the Retrieval Augmented Generation pattern - -### Docker Configuration - -The Docker setup includes: - -1. **Jupyter Container**: - - Based on minimal Jupyter notebook image - - Adds Java 21, Maven, and the JJava kernel - - Includes Python environment with PyTorch and other ML libraries - -2. **Redis Container**: - - Uses Redis Stack image with Vector Search capabilities - - Persists data using Docker volumes - - Exposes Redis on port 6379 and RedisInsight on port 8001 - -## Example Applications +## Example Notebooks & Applications ### Beer Recommendation Chatbot @@ -146,3 +44,11 @@ The `spring-ai-rag.ipynb` notebook demonstrates: - Connecting to OpenAI for LLM capabilities - Building a RAG pipeline to answer beer-related queries - Semantic search over beer properties and descriptions + +### Vector Similarity Search with Redis OM Spring and Spring Boot + +The `spring_boot_redis_om_spring` directory contains a Spring Boot application that demonstrates how to use Redis OM Spring for vector similarity search. The application allows you to: +- Add movies to the Redis database +- Search for movies based on semantic similarity on the synopsis of the movie +- Perform hybrid search by adding filters to genre, cast, and year + diff --git a/java-recipes/applications/vector-similarity-search/spring_boot_redis_om_spring.md b/java-recipes/applications/vector-similarity-search/spring_boot_redis_om_spring.md new file mode 100644 index 00000000..db8d6d4f --- /dev/null +++ b/java-recipes/applications/vector-similarity-search/spring_boot_redis_om_spring.md @@ -0,0 +1,358 @@ +# Vector Similarity Search with Redis OM Spring (Spring Boot) + +![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120) + +This guide demonstrates how to build a Vector Similarity Search (VSS) system using Spring Boot and Redis OM Spring. The example allows movies to be searched by their synopses based on semantic similarity rather than keyword matching. + +## Prerequisites + +- Java 21 +- Maven for dependency management +- Docker and Docker Compose (for running Redis) +- OpenAI API key for text embeddings + +## Repository + +The repository for this demo can be found [here](https://github.com/redis/redis-om-spring/tree/main/demos/roms-vss-movies) + +## Project Structure Overview + +The project implements a system that demonstrates vector similarity search using Redis 8's built-in capabilities together with Redis OM Spring: + +The repository can be found at: + +``` +roms-vss-movies/ +├── src/main/java/dev/raphaeldelio/redis8demovectorsimilaritysearch/ +│ ├── controller/ +│ │ └── SearchController.java # REST endpoints for search +│ ├── domain/ +│ │ └── Movie.java # Entity with vector annotations +│ ├── repository/ +│ │ └── MovieRepository.java # Redis repository interface +│ ├── service/ +│ │ ├── MovieService.java # Service for data loading +│ │ └── SearchService.java # Service for vector search +│ └── RomsVectorSimilaritySearchMovies.java # Main application +└── src/main/resources/ + ├── application.properties # Application configuration + └── movies.json # Sample dataset +``` + +## Getting Started + +1. **Start the Redis instance**: + ```bash + docker-compose up -d redis-vector-search + ``` + +2. **Build and run the application**: + ```bash + mvn spring-boot:run + ``` + +## Dependencies + +This application uses the following key dependencies: + +```xml + + + com.redis.om.spring + redis-om-spring + 0.9.11 + + + + + org.springframework.ai + spring-ai-openai + 1.0.0-M6 + + + org.springframework.ai + spring-ai-transformers + 1.0.0-M6 + +``` + +## Implementation Details + +### 1. Define the `Movie` entity + +Redis OM Spring provides two annotations that makes it easy to vectorize data and perform vector similarity search from within Spring Boot. +- `@Vectorize`: Automatically generates vector embeddings from the text field +- `@Indexed`: Enables vector indexing on the field for efficient search + +The core of the implementation is the `Movie` class with Redis vector indexing annotations: + +```java +@RedisHash // This annotation is used by Redis OM Spring to store the entity as a hash in Redis +public class Movie { + + @Id // IDs are automatically generated by Redis OM Spring as ULID + private String title; + + @Indexed(sortable = true) // This annotation enables indexing on the field for filtering and sorting + private int year; + + @Indexed + private List cast; + + @Indexed + private List genres; + + private String href; + + // This annotation automatically generates vector embeddings from the text + @Vectorize( + destination = "embeddedExtract", // The field where the embedding will be stored + embeddingType = EmbeddingType.SENTENCE, // Type of embedding to generate (Sentence, Image, face, or word) + provider = EmbeddingProvider.OPENAI, // The provider for generating embeddings (OpenAI, Transformers, VertexAI, etc.) + openAiEmbeddingModel = OpenAiApi.EmbeddingModel.TEXT_EMBEDDING_3_LARGE // The specific OpenAI model to use for embeddings + ) + private String extract; + + // This defines the vector field that will store the embeddings + // The indexed annotation enables vector search on this field + @Indexed( + schemaFieldType = SchemaFieldType.VECTOR, // Defines the field type as a vector + algorithm = VectorField.VectorAlgorithm.FLAT, // The algorithm used for vector search (FLAT or HNSW) + type = VectorType.FLOAT32, + dimension = 3072, // The dimension of the vector (must match the embedding model) + distanceMetric = DistanceMetric.COSINE, // The distance metric used for similarity search (Cosine or Euclidean) + initialCapacity = 10 + ) + private byte[] embeddedExtract; + + private String thumbnail; + private int thumbnailWidth; + private int thumbnailHeight; + + // Getters and setters... +} +``` + +### 2. Repository Interface + +A simple repository interface extends `RedisEnhancedRepository`. This will be used to load the data into Redis using the saveAll() method: + +```java +public interface MovieRepository extends RedisEnhancedRepository { +} +``` + +This provides basic CRUD operations for `Movie` entities, with the first generic parameter being the entity type and the second being the ID type. + +### 3. Search Service + +The search service uses two beans provided by Redis OM Spring: +- `EntityStream`: For creating a stream of entities to perform searches. The Entity Stream must not be confused with the Java Streams API. The Entity Stream will generate a Redis Command that will be sent to Redis so that Redis can perform the searching, filtering and sorting efficiently on its side. +- `Embedder`: Used for generating the embedding for the query sent by the user. It will be generated following the configuration of the `@Vectorize` annotation defined in the `Movie` class/ + +The search functionality is implemented in the `SearchService`: + +```java +@Service +public class SearchService { + + private static final Logger logger = LoggerFactory.getLogger(SearchService.class); + private final EntityStream entityStream; + private final Embedder embedder; + + public SearchService(EntityStream entityStream, Embedder embedder) { + this.entityStream = entityStream; + this.embedder = embedder; + } + + public List> search( + String query, + Integer yearMin, + Integer yearMax, + List cast, + List genres, + Integer numberOfNearestNeighbors) { + logger.info("Received text: {}", query); + logger.info("Received yearMin: {} yearMax: {}", yearMin, yearMax); + logger.info("Received cast: {}", cast); + logger.info("Received genres: {}", genres); + + if (numberOfNearestNeighbors == null) numberOfNearestNeighbors = 3; + if (yearMin == null) yearMin = 1900; + if (yearMax == null) yearMax = 2100; + + // Convert query text to vector embedding + byte[] embeddedQuery = embedder.getTextEmbeddingsAsBytes(List.of(query), Movie$.EXTRACT).getFirst(); + + // Perform vector search with additional filters + SearchStream stream = entityStream.of(Movie.class); + return stream + // KNN search for nearest vectors + .filter(Movie$.EMBEDDED_EXTRACT.knn(numberOfNearestNeighbors, embeddedQuery)) + // Additional metadata filters (hybrid search) + .filter(Movie$.YEAR.between(yearMin, yearMax)) + .filter(Movie$.CAST.eq(cast)) + .filter(Movie$.GENRES.eq(genres)) + // Sort by similarity score + .sorted(Movie$._EMBEDDED_EXTRACT_SCORE) + // Return both the movie and its similarity score + .map(Fields.of(Movie$._THIS, Movie$._EMBEDDED_EXTRACT_SCORE)) + .collect(Collectors.toList()); + } +} +``` + +Key features of the search service: +- Uses `EntityStream` to create a search stream for `Movie` entities +- Converts the text query into a vector embedding +- Uses K-nearest neighbors (KNN) search to find similar vectors +- Applies additional filters for hybrid search (combining vector and traditional search) +- Returns pairs of movies and their similarity scores + +### 4. Movie Service for Data Loading + +The `MovieService` handles loading movie data into Redis. It reads a JSON file containing movie date and save the movies into Redis. +It may take one or two minutes to load the data for the 36 thousand movies in the file because the embedding generation is done in the background. +The `@Vectorize` annotation will generate the embeddings for the `extract` field when the movie is saved into Redis.: + +```java +@Service +public class MovieService { + + private static final Logger log = LoggerFactory.getLogger(MovieService.class); + private final ObjectMapper objectMapper; + private final ResourceLoader resourceLoader; + private final MovieRepository movieRepository; + + public MovieService(ObjectMapper objectMapper, ResourceLoader resourceLoader, MovieRepository movieRepository) { + this.objectMapper = objectMapper; + this.resourceLoader = resourceLoader; + this.movieRepository = movieRepository; + } + + public void loadAndSaveMovies(String filePath) throws Exception { + Resource resource = resourceLoader.getResource("classpath:" + filePath); + try (InputStream is = resource.getInputStream()) { + List movies = objectMapper.readValue(is, new TypeReference<>() {}); + List unprocessedMovies = movies.stream() + .filter(movie -> !movieRepository.existsById(movie.getTitle()) && + movie.getYear() > 1980 + ).toList(); + long systemMillis = System.currentTimeMillis(); + movieRepository.saveAll(unprocessedMovies); + long elapsedMillis = System.currentTimeMillis() - systemMillis; + log.info("Saved " + movies.size() + " movies in " + elapsedMillis + " ms"); + } + } + + public boolean isDataLoaded() { + return movieRepository.count() > 0; + } +} +``` + +### 5. Search Controller + +The REST controller exposes the search endpoint: + +```java +@RestController +public class SearchController { + + private final SearchService searchService; + + public SearchController(SearchService searchService) { + this.searchService = searchService; + } + + @GetMapping("/search") + public Map search( + @RequestParam(required = false) String text, + @RequestParam(required = false) Integer yearMin, + @RequestParam(required = false) Integer yearMax, + @RequestParam(required = false) List cast, + @RequestParam(required = false) List genres, + @RequestParam(required = false) Integer numberOfNearestNeighbors + ) { + List> matchedMovies = searchService.search( + text, + yearMin, + yearMax, + cast, + genres, + numberOfNearestNeighbors + ); + return Map.of( + "matchedMovies", matchedMovies, + "count", matchedMovies.size() + ); + } +} +``` + +### 6. Application Bootstrap + +The main application class initializes Redis OM Spring and loads data: + +```java +@SpringBootApplication +@EnableRedisEnhancedRepositories(basePackages = {"dev.raphaeldelio.redis8demo*"}) +public class Redis8DemoVectorSimilaritySearchApplication { + + public static void main(String[] args) { + SpringApplication.run(Redis8DemoVectorSimilaritySearchApplication.class, args); + } + + @Bean + CommandLineRunner loadData(MovieService movieService) { + return args -> { + if (movieService.isDataLoaded()) { + System.out.println("Data already loaded. Skipping data load."); + return; + } + movieService.loadAndSaveMovies("movies.json"); + }; + } +} +``` + +The `@EnableRedisEnhancedRepositories` annotation activates Redis OM Spring's repository support. + +## Example API Requests + +You can make requests to the search endpoint: + +``` +GET http://localhost:8082/search?text=A movie about a young boy who goes to a wizardry school + +GET http://localhost:8082/search?numberOfNearestNeighbors=1&yearMin=1970&yearMax=1990&text=A movie about a kid and a scientist who go back in time + +GET http://localhost:8082/search?cast=Dee Wallace,Henry Thomas&text=A boy who becomes friend with an alien +``` + +## Sample Response + +```json +{ + "count": 1, + "matchedMovies": [ + { + "first": { + "title": "Back to the Future", + "year": 1985, + "cast": [ + "Michael J. Fox", + "Christopher Lloyd" + ], + "genres": [ + "Science Fiction" + ], + "extract": "Back to the Future is a 1985 American science fiction film directed by Robert Zemeckis and written by Zemeckis, and Bob Gale. It stars Michael J. Fox, Christopher Lloyd, Lea Thompson, Crispin Glover, and Thomas F. Wilson. Set in 1985, it follows Marty McFly (Fox), a teenager accidentally sent back to 1955 in a time-traveling DeLorean automobile built by his eccentric scientist friend Emmett \"Doc\" Brown (Lloyd), where he inadvertently prevents his future parents from falling in love – threatening his own existence – and is forced to reconcile them and somehow get back to the future.", + "thumbnail": "https://upload.wikimedia.org/wikipedia/en/d/d2/Back_to_the_Future.jpg" + }, + "second": 0.463297247887 + } + ] +} +``` + diff --git a/java-recipes/RAG/spring_ai_redis_rag.ipynb b/java-recipes/notebooks/RAG/spring_ai_redis_rag.ipynb similarity index 100% rename from java-recipes/RAG/spring_ai_redis_rag.ipynb rename to java-recipes/notebooks/RAG/spring_ai_redis_rag.ipynb diff --git a/java-recipes/notebooks/README.md b/java-recipes/notebooks/README.md new file mode 100644 index 00000000..a5a240e8 --- /dev/null +++ b/java-recipes/notebooks/README.md @@ -0,0 +1,136 @@ +
+
+

Redis AI Java Resources

+
+ +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +![Java](https://img.shields.io/badge/Java-21-orange) +![Spring AI](https://img.shields.io/badge/Spring%20AI-1.0.0--M6-green) + +
+
+ ✨ Java-based code examples, notebooks, and resources for using Redis in AI and ML applications. ✨ +
+ +
+
+ +[**Setup**](#setup) | [**Running the Project**](#running-the-project) | [**Notebooks**](#notebooks) | [**Project Structure**](#project-structure) | [**Implementation Details**](#implementation-details) + +
+
+ +## Setup + +This project uses Docker Compose to set up a complete environment for running Java-based AI applications with Redis. The environment includes: + +- A Jupyter Notebook server with Java kernel support +- Redis Stack (includes Redis and RedisInsight) +- Pre-installed dependencies for AI/ML workloads + +### Prerequisites + +- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) +- OpenAI API key (for notebooks that use OpenAI services) + +### Environment Configuration + +1. Create a `.env` file in the project root with your OpenAI API key: + +```bash +OPENAI_API_KEY=your_openai_api_key_here +``` + +## Running the Project + +1. Clone the repository (if you haven't already): + + ```bash + git clone https://github.com/redis-developer/redis-ai-resources.git + cd redis-ai-resources/java-resources + ``` + +2. Start the Docker containers: + + ```bash + docker-compose up -d + ``` + +3. Access the Jupyter environment: + - Open your browser and navigate to [http://localhost:8888](http://localhost:8888) + - The token is usually shown in the docker-compose logs. You can view them with: + + ```bash + docker-compose logs jupyter + ``` + +4. Access RedisInsight: + - Open your browser and navigate to [http://localhost:8001](http://localhost:8001) + - Connect to Redis using the following details: + - Host: redis-java + - Port: 6379 + - No password (unless configured) + +5. When finished, stop the containers: + + ```bash + docker-compose down + ``` + +## Notebooks + +| Notebook | Description | +| --- | --- | +| [RAG/spring_ai_redis_rag.ipynb](./RAG/spring_ai_redis_rag.ipynb) | Demonstrates building a RAG-based beer recommendation chatbot using Spring AI and Redis as the vector store | + +## Project Structure + +```bash +notebooks/ +├── .env # Environment variables (create this) +├── docker-compose.yml # Docker Compose configuration +├── jupyter/ # Jupyter configuration files +│ ├── Dockerfile # Dockerfile for Jupyter with Java kernel +│ ├── environment.yml # Conda environment specification +│ ├── install.py # JJava kernel installation script +│ ├── kernel.json # Kernel specification +│ └── java/ # Java dependencies and configuration +│ └── pom.xml # Maven project file with dependencies +└── resources/ # Data files for notebooks + └── beers.json.gz # Compressed beer dataset +``` + +## Implementation Details + +### Java Jupyter Kernel + +The project uses [JJava](https://github.com/dflib/jjava), a Jupyter kernel for Java based on JShell. This allows for interactive Java development in Jupyter notebooks. + +Key components: + +- Java 21 for modern Java features +- Maven for dependency management +- JJava kernel for Jupyter integration + +### Spring AI Integration + +The Spring AI notebooks showcase how to use Spring's AI capabilities with Redis: + +- **Spring AI**: Framework for building AI-powered applications +- **Redis Vector Store**: Used for storing and querying vector embeddings +- **Transformer Models**: For generating embeddings locally +- **RAG Pattern**: Demonstrates the Retrieval Augmented Generation pattern + +### Docker Configuration + +The Docker setup includes: + +1. **Jupyter Container**: + - Based on minimal Jupyter notebook image + - Adds Java 21, Maven, and the JJava kernel + - Includes Python environment with PyTorch and other ML libraries + +2. **Redis Container**: + - Uses Redis Stack image with Vector Search capabilities + - Persists data using Docker volumes + - Exposes Redis on port 6379 and RedisInsight on port 8001 \ No newline at end of file diff --git a/java-recipes/docker-compose.yml b/java-recipes/notebooks/docker-compose.yml similarity index 100% rename from java-recipes/docker-compose.yml rename to java-recipes/notebooks/docker-compose.yml diff --git a/java-recipes/jupyter/Dockerfile b/java-recipes/notebooks/jupyter/Dockerfile similarity index 100% rename from java-recipes/jupyter/Dockerfile rename to java-recipes/notebooks/jupyter/Dockerfile diff --git a/java-recipes/jupyter/environment.yml b/java-recipes/notebooks/jupyter/environment.yml similarity index 100% rename from java-recipes/jupyter/environment.yml rename to java-recipes/notebooks/jupyter/environment.yml diff --git a/java-recipes/jupyter/install.py b/java-recipes/notebooks/jupyter/install.py similarity index 98% rename from java-recipes/jupyter/install.py rename to java-recipes/notebooks/jupyter/install.py index 70cf75f4..78bca62c 100644 --- a/java-recipes/jupyter/install.py +++ b/java-recipes/notebooks/jupyter/install.py @@ -1,11 +1,11 @@ -import argparse -import json -import os -import sys - -from jupyter_client.kernelspec import KernelSpecManager - -ALIASES = { +import argparse +import json +import os +import sys + +from jupyter_client.kernelspec import KernelSpecManager + +ALIASES = { "IJAVA_CLASSPATH": { }, "IJAVA_COMPILER_OPTS": { @@ -17,93 +17,93 @@ "IJAVA_TIMEOUT": { "NO_TIMEOUT": "-1", }, - -} - -NAME_MAP = { + +} + +NAME_MAP = { "classpath": "IJAVA_CLASSPATH", "comp-opts": "IJAVA_COMPILER_OPTS", "startup-scripts-path": "IJAVA_STARTUP_SCRIPTS_PATH", "startup-script": "IJAVA_STARTUP_SCRIPT", "timeout": "IJAVA_TIMEOUT", - -} - -def type_assertion(name, type_fn): - env = NAME_MAP[name] - aliases = ALIASES.get(env, {}) - - def checker(value): - alias = aliases.get(value, value) - type_fn(alias) - return alias - setattr(checker, '__name__', getattr(type_fn, '__name__', 'type_fn')) - return checker - -class EnvVar(argparse.Action): - def __init__(self, option_strings, dest, aliases=None, name_map=None, list_sep=None, **kwargs): - super(EnvVar, self).__init__(option_strings, dest, **kwargs) - - if aliases is None: aliases = {} - if name_map is None: name_map = {} - - self.aliases = aliases - self.name_map = name_map - self.list_sep = list_sep - - for name in self.option_strings: - if name.lstrip('-') not in name_map: - raise ValueError('Name "%s" is not mapped to an environment variable' % name.lstrip('-')) - - - def __call__(self, parser, namespace, value, option_string=None): - if option_string is None: - raise ValueError('option_string is required') - - env = getattr(namespace, self.dest, None) - if env is None: - env = {} - - name = option_string.lstrip('-') - env_var = self.name_map[name] - - if self.list_sep: - old = env.get(env_var) - value = old + self.list_sep + str(value) if old is not None else str(value) - - env[env_var] = value - - setattr(namespace, self.dest, env) - -if __name__ == '__main__': - parser = argparse.ArgumentParser(description='Install the java kernel.') - - install_location = parser.add_mutually_exclusive_group() - install_location.add_argument( - '--user', - help='Install to the per-user kernel registry.', - action='store_true' - ) - install_location.add_argument( - '--sys-prefix', - help="Install to Python's sys.prefix. Useful in conda/virtual environments.", - action='store_true' - ) - install_location.add_argument( - '--prefix', - help=''' - Specify a prefix to install to, e.g. an env. - The kernelspec will be installed in PREFIX/share/jupyter/kernels/ - ''', - default='' - ) - - parser.add_argument( - '--replace', - help='Replace any existing kernel spec with this name.', - action='store_true' - ) - + +} + +def type_assertion(name, type_fn): + env = NAME_MAP[name] + aliases = ALIASES.get(env, {}) + + def checker(value): + alias = aliases.get(value, value) + type_fn(alias) + return alias + setattr(checker, '__name__', getattr(type_fn, '__name__', 'type_fn')) + return checker + +class EnvVar(argparse.Action): + def __init__(self, option_strings, dest, aliases=None, name_map=None, list_sep=None, **kwargs): + super(EnvVar, self).__init__(option_strings, dest, **kwargs) + + if aliases is None: aliases = {} + if name_map is None: name_map = {} + + self.aliases = aliases + self.name_map = name_map + self.list_sep = list_sep + + for name in self.option_strings: + if name.lstrip('-') not in name_map: + raise ValueError('Name "%s" is not mapped to an environment variable' % name.lstrip('-')) + + + def __call__(self, parser, namespace, value, option_string=None): + if option_string is None: + raise ValueError('option_string is required') + + env = getattr(namespace, self.dest, None) + if env is None: + env = {} + + name = option_string.lstrip('-') + env_var = self.name_map[name] + + if self.list_sep: + old = env.get(env_var) + value = old + self.list_sep + str(value) if old is not None else str(value) + + env[env_var] = value + + setattr(namespace, self.dest, env) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description='Install the java kernel.') + + install_location = parser.add_mutually_exclusive_group() + install_location.add_argument( + '--user', + help='Install to the per-user kernel registry.', + action='store_true' + ) + install_location.add_argument( + '--sys-prefix', + help="Install to Python's sys.prefix. Useful in conda/virtual environments.", + action='store_true' + ) + install_location.add_argument( + '--prefix', + help=''' + Specify a prefix to install to, e.g. an env. + The kernelspec will be installed in PREFIX/share/jupyter/kernels/ + ''', + default='' + ) + + parser.add_argument( + '--replace', + help='Replace any existing kernel spec with this name.', + action='store_true' + ) + parser.add_argument( "--classpath", dest="env", @@ -152,46 +152,46 @@ def __call__(self, parser, namespace, value, option_string=None): help="A duration specifying a timeout (in milliseconds by default) for a _single top level statement_. If less than `1` then there is no timeout. If desired a time may be specified with a `TimeUnit` may be given following the duration number (ex `\"30 SECONDS\"`).", type=type_assertion("timeout", str), ) - - - args = parser.parse_args() - + + + args = parser.parse_args() + if not hasattr(args, "env") or getattr(args, "env") is None: setattr(args, "env", {}) - - - # Install the kernel - install_dest = KernelSpecManager().install_kernel_spec( - os.path.join(os.path.dirname(os.path.abspath(__file__)), 'java'), - kernel_name='java', - user=args.user, - prefix=sys.prefix if args.sys_prefix else args.prefix, - replace=args.replace - ) - - # Connect the self referencing token left in the kernel.json to point to it's install location. - - # Prepare the token replacement string which should be properly escaped for use in a JSON string - # The [1:-1] trims the first and last " json.dumps adds for strings. - install_dest_json_fragment = json.dumps(install_dest)[1:-1] - - # Prepare the paths to the installed kernel.json and the one bundled with this installer. - local_kernel_json_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'java', 'kernel.json') - installed_kernel_json_path = os.path.join(install_dest, 'kernel.json') - - # Replace the @KERNEL_INSTALL_DIRECTORY@ token with the path to where the kernel was installed - # in the installed kernel.json from the local template. - with open(local_kernel_json_path, 'r') as template_kernel_json_file: - template_kernel_json_contents = template_kernel_json_file.read() - kernel_json_contents = template_kernel_json_contents.replace( - '@KERNEL_INSTALL_DIRECTORY@', - install_dest_json_fragment - ) - kernel_json_json_contents = json.loads(kernel_json_contents) - kernel_env = kernel_json_json_contents.setdefault('env', {}) - for k, v in args.env.items(): - kernel_env[k] = v - with open(installed_kernel_json_path, 'w') as installed_kernel_json_file: - json.dump(kernel_json_json_contents, installed_kernel_json_file, indent=4, sort_keys=True) - - print('Installed java kernel into "%s"' % install_dest) + + + # Install the kernel + install_dest = KernelSpecManager().install_kernel_spec( + os.path.join(os.path.dirname(os.path.abspath(__file__)), 'java'), + kernel_name='java', + user=args.user, + prefix=sys.prefix if args.sys_prefix else args.prefix, + replace=args.replace + ) + + # Connect the self referencing token left in the kernel.json to point to it's install location. + + # Prepare the token replacement string which should be properly escaped for use in a JSON string + # The [1:-1] trims the first and last " json.dumps adds for strings. + install_dest_json_fragment = json.dumps(install_dest)[1:-1] + + # Prepare the paths to the installed kernel.json and the one bundled with this installer. + local_kernel_json_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'java', 'kernel.json') + installed_kernel_json_path = os.path.join(install_dest, 'kernel.json') + + # Replace the @KERNEL_INSTALL_DIRECTORY@ token with the path to where the kernel was installed + # in the installed kernel.json from the local template. + with open(local_kernel_json_path, 'r') as template_kernel_json_file: + template_kernel_json_contents = template_kernel_json_file.read() + kernel_json_contents = template_kernel_json_contents.replace( + '@KERNEL_INSTALL_DIRECTORY@', + install_dest_json_fragment + ) + kernel_json_json_contents = json.loads(kernel_json_contents) + kernel_env = kernel_json_json_contents.setdefault('env', {}) + for k, v in args.env.items(): + kernel_env[k] = v + with open(installed_kernel_json_path, 'w') as installed_kernel_json_file: + json.dump(kernel_json_json_contents, installed_kernel_json_file, indent=4, sort_keys=True) + + print('Installed java kernel into "%s"' % install_dest) diff --git a/java-recipes/jupyter/java/kernel.json b/java-recipes/notebooks/jupyter/java/kernel.json similarity index 100% rename from java-recipes/jupyter/java/kernel.json rename to java-recipes/notebooks/jupyter/java/kernel.json diff --git a/java-recipes/jupyter/java/pom.xml b/java-recipes/notebooks/jupyter/java/pom.xml similarity index 100% rename from java-recipes/jupyter/java/pom.xml rename to java-recipes/notebooks/jupyter/java/pom.xml diff --git a/java-recipes/resources/beers.json.gz b/java-recipes/notebooks/resources/beers.json.gz similarity index 100% rename from java-recipes/resources/beers.json.gz rename to java-recipes/notebooks/resources/beers.json.gz