diff --git a/README.md b/README.md index c954a8ee..a01de17f 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,7 @@ Need quickstarts to begin your Redis AI journey? | 🔄 **Hybrid Search** - Hybrid search techniques with Redis (BM25 + Vector) | [![Open In GitHub](https://img.shields.io/badge/View-GitHub-green)](python-recipes/vector-search/02_hybrid_search.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/02_hybrid_search.ipynb) | | 🔢 **Data Type Support** - Shows how to convert a float32 index to float16 or integer dataypes | [![Open In GitHub](https://img.shields.io/badge/View-GitHub-green)](python-recipes/vector-search/03_dtype_support.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/03_dtype_support.ipynb) | | 📊 **Benchmarking Basics** - Overview of search benchmarking basics with RedisVL and Python multiprocessing | [![Open In GitHub](https://img.shields.io/badge/View-GitHub-green)](python-recipes/vector-search/04_redisvl_benchmarking_basics.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/04_redisvl_benchmarking_basics.ipynb) | +| 📊 **Multi Vector Search** - Overview of multi vector queries with RedisVL | [![Open In GitHub](https://img.shields.io/badge/View-GitHub-green)](python-recipes/vector-search/05_multivector_search.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/05_multivector_search.ipynb) | ### Retrieval Augmented Generation (RAG) diff --git a/python-recipes/vector-search/05_multivector_search.ipynb b/python-recipes/vector-search/05_multivector_search.ipynb new file mode 100644 index 00000000..65541b4b --- /dev/null +++ b/python-recipes/vector-search/05_multivector_search.ipynb @@ -0,0 +1,548 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120)\n", + "# Implementing multi vector search with Redis\n", + "\n", + "Multi vector search is the ability to combine the scores of multiple different vector similarity values to determine relevancy. This notebook will cover how to define a multi vector index and execute multi vector queries.\n", + "\n", + "## Let's Begin!\n", + "\"Open\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install Packages" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install \"redisvl>=0.10.0\"" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n", + "To disable this warning, you can either:\n", + "\t- Avoid using `tokenizers` before the fork if possible\n", + "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Found existing installation: redisvl 0.9.1\n", + "Uninstalling redisvl-0.9.1:\n", + " Successfully uninstalled redisvl-0.9.1\n", + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip uninstall -y redisvl" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'0.10.0'" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# check version\n", + "import redisvl\n", + "\n", + "redisvl.__version__" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Data/Index Preparation\n", + "\n", + "In this section:\n", + "\n", + "1. We prepare the data necessary for our multi vector search implementations by loading a collection of movies. Each movie object contains the following attributes:\n", + " - `title`\n", + " - `rating`\n", + " - `description`\n", + " - `genre`\n", + "\n", + "2. We'll generate vector embeddings from the movie descriptions. We'll use different models and generate multiple different vectors for each movie.\n", + "\n", + "3. After preparing the data, we populate a search index with these movie records, each with multiple vectors." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Running remotely or in collab? Run this cell to download the necessary dataset." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# NBVAL_SKIP\n", + "!git clone https://github.com/redis-developer/redis-ai-resources.git temp_repo\n", + "!mv temp_repo/python-recipes/vector-search/resources .\n", + "!rm -rf temp_repo" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install Redis Stack\n", + "\n", + "Later in this tutorial, Redis will be used to store, index, and query vector\n", + "embeddings and full text fields. **We need to have a Redis\n", + "instance available.**\n", + "\n", + "#### Local Redis\n", + "Use the shell script below to download, extract, and install [Redis Stack](https://redis.io/docs/getting-started/install-stack/) directly from the Redis package archive." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# NBVAL_SKIP\n", + "%%sh\n", + "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n", + "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n", + "sudo apt-get update > /dev/null 2>&1\n", + "sudo apt-get install redis-stack-server > /dev/null 2>&1\n", + "redis-stack-server --daemonize yes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Alternative Redis Access (Cloud, Docker, other)\n", + "There are many ways to get the necessary redis-stack instance running\n", + "1. On cloud, deploy a [FREE instance of Redis in the cloud](https://redis.com/try-free/). Or, if you have your\n", + "own version of Redis Enterprise running, that works too!\n", + "2. Per OS, [see the docs](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)\n", + "3. With docker: `docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Define the Redis Connection URL\n", + "\n", + "By default this notebook connects to the local instance of Redis Stack. **If you have your own Redis Enterprise instance** - replace REDIS_PASSWORD, REDIS_HOST and REDIS_PORT values with your own." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "# Replace values below with your own if using Redis Cloud instance\n", + "REDIS_HOST = os.getenv(\"REDIS_HOST\", \"localhost\") # ex: \"redis-18374.c253.us-central1-1.gce.cloud.redislabs.com\"\n", + "REDIS_PORT = os.getenv(\"REDIS_PORT\", \"6379\") # ex: 18374\n", + "REDIS_PASSWORD = os.getenv(\"REDIS_PASSWORD\", \"\") # ex: \"1TNxTEdYRDgIDKM2gDfasupCADXXXX\"\n", + "\n", + "# If SSL is enabled on the endpoint, use rediss:// as the URL prefix\n", + "REDIS_URL = f\"redis://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create redis client, load data, generate embeddings" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from redis import Redis\n", + "\n", + "client = Redis.from_url(REDIS_URL)\n", + "client.ping()" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "\n", + "with open(\"resources/movies.json\", 'r') as file:\n", + " movies = json.load(file)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Multi-Vector Index with Multiple Embedding Models\n", + "\n", + "Now let's create a multi-vector search setup by using multiple embedding models for different aspects of our movie data. This approach allows us to:\n", + "\n", + "1. **Use specialized embeddings** - Different models optimized for different tasks\n", + "2. **Combine multiple perspectives** - Search across different semantic representations\n", + "3. **Improve search quality** - Generate embeddings from different sections of your data\n", + "\n", + "We'll create a new index with multiple vector fields and demonstrate how to query across them." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "14:45:04 datasets INFO PyTorch version 2.3.0 available.\n", + "14:45:05 sentence_transformers.SentenceTransformer INFO Use pytorch device_name: mps\n", + "14:45:05 sentence_transformers.SentenceTransformer INFO Load pretrained SentenceTransformer: sentence-transformers/all-MiniLM-L6-v2\n", + "14:45:07 sentence_transformers.SentenceTransformer INFO Use pytorch device_name: mps\n", + "14:45:07 sentence_transformers.SentenceTransformer INFO Load pretrained SentenceTransformer: sentence-transformers/all-mpnet-base-v2\n", + "14:45:08 sentence_transformers.SentenceTransformer INFO Use pytorch device_name: mps\n", + "14:45:08 sentence_transformers.SentenceTransformer INFO Load pretrained SentenceTransformer: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2\n" + ] + } + ], + "source": [ + "from redisvl.utils.vectorize import HFTextVectorizer\n", + "from redisvl.extensions.cache.embeddings import EmbeddingsCache\n", + "\n", + "# Model 1: General purpose embeddings (what we used before)\n", + "general_model = HFTextVectorizer(\n", + " model='sentence-transformers/all-MiniLM-L6-v2',\n", + " cache=EmbeddingsCache(\n", + " name=\"embedcache_general\",\n", + " ttl=600,\n", + " redis_client=client,\n", + " ),\n", + " dtype=\"float64\"\n", + ")\n", + "\n", + "# Model 2: A different model that captures different aspects of the description data\n", + "movie_model = HFTextVectorizer(\n", + " model='sentence-transformers/all-mpnet-base-v2',\n", + " cache=EmbeddingsCache(\n", + " name=\"embedcache_movie\",\n", + " ttl=600,\n", + " redis_client=client,\n", + " ),\n", + " dtype=\"float32\"\n", + ")\n", + "\n", + "# Model 3: Genre-focused embeddings for better genre understanding\n", + "genre_model = HFTextVectorizer(\n", + " model='sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2',\n", + " cache=EmbeddingsCache(\n", + " name=\"embedcache_genre\",\n", + " ttl=600,\n", + " redis_client=client,\n", + " ),\n", + " dtype=\"float32\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's highlight the flexibility of multi-vector search in RedisVL:\n", + "- Different embedding models can be used for each field. Here we have 3 different HuggingFace vectorizors, but you can mix and match any vectorizer.\n", + "- These models can have different dimensions and datatypes. Want to use a large model with high fidelity and pair it with a lightweight model? Sure thing.\n", + "- The data source for these embeddings can be anything. In the cell below we generate embeddings from different aspects of our data. You can have a multiple text embeddings and image embeddings on the same document.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Generating multiple embeddings for movies...\n", + "Generated embeddings for 20 movies\n" + ] + } + ], + "source": [ + "# Generate multiple embeddings for each movie\n", + "print(\"Generating multiple embeddings for movies...\")\n", + "\n", + "multi_vector_movie_data = []\n", + "for movie in movies:\n", + " movie_with_vectors = {\n", + " **movie,\n", + " \"description_vector_general\": general_model.embed(movie[\"description\"], as_buffer=True),\n", + " \"description_vector_movie\": movie_model.embed(movie[\"description\"], as_buffer=True),\n", + " \"description_vector_genre\": genre_model.embed(f\"{movie['genre']} {movie['description']}\", as_buffer=True),\n", + " }\n", + " multi_vector_movie_data.append(movie_with_vectors)\n", + "\n", + "print(f\"Generated embeddings for {len(multi_vector_movie_data)} movies\")" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Multi-vector index created and populated successfully!\n" + ] + } + ], + "source": [ + "from redisvl.schema import IndexSchema\n", + "from redisvl.index import SearchIndex\n", + "\n", + "# Create a multi-vector index schema\n", + "multi_vector_schema = IndexSchema.from_dict({\n", + " \"index\": {\n", + " \"name\": \"movies_multivector\",\n", + " \"prefix\": \"movie_mv\",\n", + " \"storage\": \"hash\"\n", + " },\n", + " \"fields\": [\n", + " { \"name\": \"title\", \"type\": \"text\" },\n", + " { \"name\": \"description\", \"type\": \"text\" },\n", + " { \"name\": \"genre\", \"type\": \"tag\", \"attrs\": {\"sortable\": True}},\n", + " { \"name\": \"rating\", \"type\": \"numeric\", \"attrs\": {\"sortable\": True}},\n", + " {\n", + " \"name\": \"description_vector_general\",\n", + " \"type\": \"vector\",\n", + " \"attrs\": {\n", + " \"dims\": 384,\n", + " \"distance_metric\": \"cosine\",\n", + " \"algorithm\": \"hnsw\",\n", + " \"datatype\": \"float64\"\n", + " }\n", + " },\n", + " {\n", + " \"name\": \"description_vector_movie\",\n", + " \"type\": \"vector\",\n", + " \"attrs\": {\n", + " \"dims\": 768,\n", + " \"distance_metric\": \"cosine\",\n", + " \"algorithm\": \"hnsw\",\n", + " \"datatype\": \"float32\"\n", + " }\n", + " },\n", + " {\n", + " \"name\": \"description_vector_genre\",\n", + " \"type\": \"vector\",\n", + " \"attrs\": {\n", + " \"dims\": 384,\n", + " \"distance_metric\": \"cosine\",\n", + " \"algorithm\": \"hnsw\",\n", + " \"datatype\": \"float32\"\n", + " }\n", + " },\n", + " ]\n", + "})\n", + "\n", + "# Create the multi-vector index\n", + "multi_vector_index = SearchIndex(multi_vector_schema, client, validate_on_load=True)\n", + "multi_vector_index.create(overwrite=True, drop=True)\n", + "\n", + "# Load the multi-vector data\n", + "multi_vector_index.load(multi_vector_movie_data)\n", + "print(\"Multi-vector index created and populated successfully!\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice how each vector field in our index has its own definition with its own attributes.\n", + "When Constructing an index that is intended for use with MultiVectorQuery each vector field may have different `datatype` `dims` and `algorithm`, but all must have a `cosine` `distance_metric` in order to properly compute the relative weighting." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Using the Official MultiVectorQuery Class\n", + "\n", + "Now let's demonstrate how to run a `MultiVectorQuery` in RedisVL. This class provides a clean, way to perform multi-vector search with weighted combinations. It utilizes the `Vector` class to contain the individual query vectors, which is a departure in syntax from other RedisVL queries." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1. The Incredibles \n", + " Genre: comedy, Rating: 8\n", + " Description: A family of undercover superheroes, while trying to live the quiet suburban life, are forced into ac...\n", + "\n", + "2. The Avengers \n", + " Genre: action, Rating: 8\n", + " Description: Earth's mightiest heroes come together to stop an alien invasion that threatens the entire planet....\n", + "\n", + "3. Mad Max: Fury Road \n", + " Genre: action, Rating: 8\n", + " Description: In a post-apocalyptic wasteland, Max teams up with Furiosa to escape a tyrant's clutches and find fr...\n", + "\n", + "4. John Wick \n", + " Genre: action, Rating: 8\n", + " Description: A retired hitman seeks vengeance against those who wronged him, leaving a trail of destruction in hi...\n", + "\n", + "5. The Dark Knight \n", + " Genre: action, Rating: 9\n", + " Description: Batman faces off against the Joker, a criminal mastermind who threatens to plunge Gotham into chaos....\n", + "\n" + ] + } + ], + "source": [ + "from redisvl.query import Vector, MultiVectorQuery\n", + "query_text = \"action movie with superheroes and explosions\"\n", + "\n", + "num_results = 5\n", + "\n", + "# Create Vector objects for each embedding model\n", + "query_vectors = [\n", + " Vector(\n", + " vector=general_model.embed(query_text, as_buffer=True),\n", + " field_name=\"description_vector_general\",\n", + " dtype=\"float64\",\n", + " weight=0.3 # 30% weight for general embeddings\n", + " ),\n", + " Vector(\n", + " vector=movie_model.embed(query_text, as_buffer=True),\n", + " field_name=\"description_vector_movie\",\n", + " dtype=\"float32\",\n", + " weight=0.5 # 50% weight for movie-specific embeddings\n", + " ),\n", + " Vector(\n", + " vector=genre_model.embed(f\"{query_text}\", as_buffer=True),\n", + " field_name=\"description_vector_genre\",\n", + " dtype=\"float32\",\n", + " weight=0.2 # 20% weight for genre-focused embeddings\n", + " )\n", + "]\n", + "\n", + "query = MultiVectorQuery(\n", + " vectors=query_vectors,\n", + " num_results=num_results,\n", + " return_fields=[\"title\", \"description\", \"genre\", \"rating\"],\n", + ")\n", + "\n", + "results = multi_vector_index.query(query)\n", + "\n", + "for i, result in enumerate(results, 1):\n", + " print(f\"{i}. {result['title']} \")\n", + " print(f\" Genre: {result['genre']}, Rating: {result['rating']}\")\n", + " print(f\" Description: {result['description'][:100]}...\")\n", + " print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Wrap up\n", + "That's a wrap! Hopefully from this you were able to learn how to implement multi-vector search with multiple embedding models" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "# clean up!\n", + "multi_vector_index.delete()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "3.11.9", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}