diff --git a/README.md b/README.md index 3a015bd193..5b0addc069 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Redis Docs -OPEN SOURCE LICENSE VS. TRADEMARKS. The three-clause BSD license gives you the right to redistribute and use the software in source and binary forms, with or without modification, under certain conditions. However, open source licenses like the three-clause BSD license do not address trademarks. For further details please read the [Redis Trademark Policy](https://redis.io/legal/trademark-policy/). +OPEN SOURCE LICENSE VS. TRADEMARKS. The three-clause BSD license gives you the right to redistribute and use the software in source and binary forms, with or without modification, under certain conditions. However, open source licenses like the three-clause BSD license do not address trademarks. For further details please read the [Redis Trademark Policy](https://redis.io/legal/trademark-policy/)." ## A note for Issue and PR submitters @@ -204,4 +204,4 @@ nil pointer evaluating page.Page.Params ``` This happened for us when Hugo executed the `bannerText` logic, which inspected all parent pages by checking if they have a banner text set. The problem was that one of the parent folders was missing an `_index.md` file, which meant the folder couldn't be identified as a page. The solution was to add an `_index.md` file to the folder that didn't have it yet. - + \ No newline at end of file diff --git a/content/develop/ai/_index.md b/content/develop/ai/index.md similarity index 52% rename from content/develop/ai/_index.md rename to content/develop/ai/index.md index 23e0bea190..6b12f0e105 100644 --- a/content/develop/ai/_index.md +++ b/content/develop/ai/index.md @@ -8,14 +8,13 @@ categories: - rc description: An overview of Redis for AI documentation linkTitle: Redis for AI -hideListLinks: true weight: 40 --- -Redis stores and indexes vector embeddings that semantically represent unstructured data including text passages, images, videos, or audio. Store vectors and the associated metadata within [hashes]({{< relref "/develop/data-types/hashes" >}}) or [JSON]({{< relref "/develop/data-types/json" >}}) documents for [indexing]({{< relref "/develop/interact/search-and-query/indexing" >}}) and [querying]({{< relref "/develop/interact/search-and-query/query" >}}). Use the [vector sets]({{< relref "/develop/data-types/vector-sets" >}}) preview data type to add items to a set, and retrieve a subset of items that are the most similar to a specified vector. +Redis stores and indexes vector embeddings that semantically represent unstructured data including text passages, images, videos, or audio. Store vectors and the associated metadata within [hashes]({{< relref "/develop/data-types/hashes" >}}) or [JSON]({{< relref "/develop/data-types/json" >}}) documents for [indexing]({{< relref "/develop/interact/search-and-query/indexing" >}}) and [querying]({{< relref "/develop/interact/search-and-query/query" >}}). | Vector | RAG | RedisVL | | :-- | :-- | :-- | -| {{AI Redis icon.}}[Redis vector database quick start guide]({{< relref "/develop/ai/vector-database" >}}) |{{AI Redis icon.}} [Retrieval-Augmented Generation quick start guide]({{< relref "/develop/ai/rag" >}}) | {{AI Redis icon.}}[Redis vector Python client library documentation]({{< relref "/integrate/redisvl/" >}}) | +| {{AI Redis icon.}}[Redis vector database quick start guide]({{< relref "/develop/get-started/vector-database" >}}) |{{AI Redis icon.}} [Retrieval-Augmented Generation quick start guide]({{< relref "/develop/get-started/rag" >}}) | {{AI Redis icon.}}[Redis vector Python client library documentation]({{< relref "/integrate/redisvl/" >}}) | #### Overview @@ -30,14 +29,10 @@ This page organized into a few sections depending on what you’re trying to do: ## How to's -[**Create a vector index**]({{< relref "/develop/ai/vector-fields/" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports `FLAT` and `HNSW` vector index types. Redis stores vectors and metadata in hashes or JSON objects. Redis supports several advanced querying strategies with vector fields including k-nearest neighbor KNN, vector range queries, and metadata filters. - -#### Learn how to index and query vector embeddings -* [redis-py (Python)]({{< relref "/develop/clients/redis-py/vecsearch" >}}) -* [NRedisStack (C#/.NET)]({{< relref "/develop/clients/dotnet/vecsearch" >}}) -* [node-redis (JavaScript)]({{< relref "/develop/clients/nodejs/vecsearch" >}}) -* [Jedis (Java)]({{< relref "/develop/clients/jedis/vecsearch" >}}) -* [go-redis (Go)]({{< relref "/develop/clients/go/vecsearch" >}}) +1. [**Create a vector index**]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#flat-index" >}}) and [`HNSW`]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#hnsw-index" >}}) vector index types. +1. [**Store and update vectors**]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#store-and-update-vectors" >}}): Redis stores vectors and metadata in hashes or JSON objects. +1. [**Search with vectors**]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#search-with-vectors" >}}): Redis supports several advanced querying strategies with vector fields including k-nearest neighbor ([KNN]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#knn-vector-search" >}})), [vector range queries]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#vector-range-queries" >}}), and [metadata filters]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#filters" >}}). +1. [**Configure vector queries at runtime**]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#runtime-query-parameters" >}}). Select the best filter mode to optimize query execution. ## Concepts @@ -49,9 +44,43 @@ Learn to perform vector search and use gateways and semantic caching in your AI/ ## Quickstarts -Get started with the following Redis Python notebook. +Quickstarts or recipes are useful when you are trying to build specific functionality. For example, you might want to do RAG with LangChain or set up LLM memory for you AI agent. Get started with the following Redis Python notebooks. + +* [The place to start if you are brand new to Redis](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/redis-intro/00_redis_intro.ipynb) + +#### Hybrid and vector search +Vector search retrieves results based on the similarity of high-dimensional numerical embeddings, while hybrid search combines this with traditional keyword or metadata-based filtering for more comprehensive results. +* [Implementing hybrid search with Redis](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/02_hybrid_search.ipynb) +* [Vector search with Redis Python client](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/00_redispy.ipynb) +* [Vector search with Redis Vector Library](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/01_redisvl.ipynb) + +#### RAG +Retrieval Augmented Generation (aka RAG) is a technique to enhance the ability of an LLM to respond to user queries. The retrieval part of RAG is supported by a vector database, which can return semantically relevant results to a user’s query, serving as contextual information to augment the generative capabilities of an LLM. +* [RAG from scratch with the Redis Vector Library](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/01_redisvl.ipynb) +* [RAG using Redis and LangChain](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/02_langchain.ipynb) +* [RAG using Redis and LlamaIndex](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/03_llamaindex.ipynb) +* [Advanced RAG with redisvl](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/04_advanced_redisvl.ipynb) +* [RAG using Redis and Nvidia](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/05_nvidia_ai_rag_redis.ipynb) +* [Utilize RAGAS framework to evaluate RAG performance](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/06_ragas_evaluation.ipynb) +* [Vector search with Azure](https://techcommunity.microsoft.com/blog/azuredevcommunityblog/vector-similarity-search-with-azure-cache-for-redis-enterprise/3822059) +* [RAG with Spring AI](https://redis.io/blog/building-a-rag-application-with-redis-and-spring-ai/) +* [RAG with Vertex AI](https://github.com/redis-developer/gcp-redis-llm-stack/tree/main) +* [Notebook for additional tips and techniques to improve RAG quality](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/04_advanced_redisvl.ipynb) + +#### Agents +AI agents can act autonomously to plan and execute tasks for the user. +* [Notebook to get started with LangGraph and agents](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/agents/00_langgraph_redis_agentic_rag.ipynb) +* [Build a collaborative movie recommendation system using Redis for data storage, CrewAI for agent-based task execution, and LangGraph for workflow management.](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/agents/01_crewai_langgraph_redis.ipynb) + +#### LLM memory +LLMs are stateless. To maintain context within a conversation chat sessions must be stored and resent to the LLM. Redis manages the storage and retrieval of chat sessions to maintain context and conversational relevance. +* [LLM session manager with semantic similarity](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/llm-session-manager/00_llm_session_manager.ipynb) +* [Handle multiple simultaneous chats with one instance](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/llm-session-manager/01_multiple_sessions.ipynb) -* [The place to start if you are brand new to Redis](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/redis-intro/00_redis_intro.ipynb). See the [Redis AI notebooks collection]({{< relref "/develop/ai/notebooks-collection" >}}) for the complete list. +#### Semantic caching +An estimated 31% of LLM queries are potentially redundant. Redis enables semantic caching to help cut down on LLM costs quickly. +* [Build a semantic cache using the Doc2Cache framework and Llama3.1](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/doc2cache_llama3_1.ipynb) +* [Build a semantic cache with Redis and Google Gemini](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/semantic_caching_gemini.ipynb) #### Computer vision Build a facial recognition system using the Facenet embedding model and RedisVL. @@ -68,13 +97,8 @@ Need a deeper-dive through different use cases and topics? * [Agentic RAG](https://github.com/redis-developer/agentic-rag) - A tutorial focused on agentic RAG with LlamaIndex and Amazon Bedrock * [RAG on Vertex AI](https://github.com/redis-developer/gcp-redis-llm-stack/tree/main) - A RAG tutorial featuring Redis with Vertex AI * [RAG workbench](https://github.com/redis-developer/redis-rag-workbench) - A development playground for exploring RAG techniques with Redis -* [Vector search with Azure](https://techcommunity.microsoft.com/blog/azuredevcommunityblog/vector-similarity-search-with-azure-cache-for-redis-enterprise/3822059) -* [RAG with Spring AI](https://redis.io/blog/building-a-rag-application-with-redis-and-spring-ai/) -* [RAG with Vertex AI](https://github.com/redis-developer/gcp-redis-llm-stack/tree/main) #### Recommendation system -* [Intro content filtering example with redisvl](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/recommendation-systems/00_content_filtering.ipynb) -* [Intro collaborative filtering example with redisvl](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/recommendation-systems/01_collaborative_filtering.ipynb) * [Recommendation systems w/ NVIDIA Merlin & Redis](https://github.com/redis-developer/redis-nvidia-recsys) - Three examples, each escalating in complexity, showcasing the process of building a realtime recsys with NVIDIA and Redis * [Redis product search](https://github.com/redis-developer/redis-product-search) - Build a real-time product search engine using features like full-text search, vector similarity, and real-time data updates diff --git a/content/develop/ai/notebooks-collection.md b/content/develop/ai/notebooks-collection.md deleted file mode 100644 index 576166fecf..0000000000 --- a/content/develop/ai/notebooks-collection.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -Title: Redis AI notebooks collection -alwaysopen: false -categories: -- docs -- operate -- rs -- rc -description: This collection showcases how Redis can be integrated into AI workflows to enhance performance, reduce latency, and enable real-time AI applications. Each notebook comes with complete code examples, explanations, and integration guides. -linkTitle: AI notebooks collection -weight: 40 ---- - - -| Notebook | Category | Description | | -|----------|----------|-------------|--| -| The place to start if you are brand new to Redis | Introduction | Great for Redis beginners looking for a guided Colab experience. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/redis-intro/00_redis_intro.ipynb) | -| Implementing hybrid search with Redis | Hybrid and Vector Search | Combines vector similarity with keyword filters. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/02_hybrid_search.ipynb) | -| Vector search with Redis Python client | Hybrid and Vector Search | Demonstrates pure vector search using the Redis Python client. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/00_redispy.ipynb) | -| Vector search with Redis Vector Library | Hybrid and Vector Search | Uses RedisVL for advanced vector indexing and querying. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/01_redisvl.ipynb) | -| RAG from scratch with Redis Vector Library | RAG | Basic RAG implementation using RedisVL. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/01_redisvl.ipynb) | -| RAG using Redis and LangChain | RAG | Shows integration between Redis and LangChain for RAG. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/02_langchain.ipynb) | -| RAG using Redis and LlamaIndex | RAG | Walkthrough of RAG with Redis and LlamaIndex. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/03_llamaindex.ipynb) | -| Advanced RAG with redisvl | RAG | Advanced concepts and techniques using RedisVL. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/04_advanced_redisvl.ipynb) | -| RAG using Redis and Nvidia | RAG | NVIDIA + Redis for LLM context retrieval. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/05_nvidia_ai_rag_redis.ipynb) | -| Utilize RAGAS framework to evaluate RAG performance | RAG | Evaluation of RAG apps using the RAGAS framework. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/06_ragas_evaluation.ipynb) | -| Vector search with Azure | RAG | Example with Azure Cache for Redis. | [Open colab](https://techcommunity.microsoft.com/blog/azuredevcommunityblog/vector-similarity-search-with-azure-cache-for-redis-enterprise/3822059) | -| RAG with Spring AI | RAG | Blog article on integrating Redis with Spring AI. | [Open colab](https://redis.io/blog/building-a-rag-application-with-redis-and-spring-ai/) | -| RAG with Vertex AI | RAG | Guide to use Redis in GCP stack with Vertex AI. | [Open colab](https://github.com/redis-developer/gcp-redis-llm-stack/tree/main) | -| LangGraph and agents | Agents | Getting started with agent workflows. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/agents/00_langgraph_redis_agentic_rag.ipynb) | -| Movie recommendation system | Agents | Collaborative agent-based movie recommender. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/agents/01_crewai_langgraph_redis.ipynb) | -| LLM session manager with semantic similarity | LLM Memory | Tracks conversation sessions using Redis and similarity. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/llm-session-manager/00_llm_session_manager.ipynb) | -| Multiple simultaneous chat sessions | LLM Memory | Supports handling multiple chat threads. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/llm-session-manager/01_multiple_sessions.ipynb) | -| Semantic cache with Llama3.1 | Semantic Caching | Uses Doc2Cache framework with Llama3.1. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/doc2cache_llama3_1.ipynb) | -| Semantic cache with Gemini | Semantic Caching | Gemini-based semantic caching demo. | [Open colab](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/semantic_caching_gemini.ipynb) | -| Facial recognition | Computer Vision | Face matching using Facenet and RedisVL. | [Open colab](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/computer-vision/00_facial_recognition_facenet.ipynb) | -| Content filtering with redisvl | Recommendation Systems | Introduction to content-based filtering. | [Open colab](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/recommendation-systems/00_content_filtering.ipynb) | -| Collaborative filtering with redisvl | Recommendation Systems | Intro to collaborative filtering with RedisVL. | [Open colab](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/recommendation-systems/01_collaborative_filtering.ipynb) | -| Advanced RAG example | Best Practices | Demonstrates mature RAG implementation patterns. | [Open colab](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/04_advanced_redisvl.ipynb) | - - diff --git a/content/develop/ai/vector-search.md b/content/develop/ai/vector-search.md deleted file mode 100644 index a283e264fb..0000000000 --- a/content/develop/ai/vector-search.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -categories: -- docs -- develop -- stack -- oss -- rs -- rc -- oss -- kubernetes -- clients -description: Query for data based on vector embeddings -linkTitle: Vector search examples -title: Vector search examples -weight: 2 ---- - -This article gives you a good overview of how to perform vector search queries with Redis Stack. See the [Redis as a vector database quick start guide]({{< relref "/develop/get-started/vector-database" >}}) for more information about Redis as a vector database. You can also find more detailed information about all the parameters in the [vector reference documentation]({{< relref "/develop/interact/search-and-query/advanced-concepts/vectors" >}}). - -A vector search query on a vector field allows you to find all vectors in a vector space that are close to a given vector. You can query for the k-nearest neighbors or vectors within a given radius. - -The examples in this article use a schema with the following fields: - -| JSON field | Field alias | Field type | Description | -| ------------------------ | ----------- | ----------- | ----------- | -| `$.description` | `description` | `TEXT` | The description of a bicycle as unstructured text | -| `$.description_embeddings` | `vector` | `VECTOR` | The vector that a machine learning model derived from the description text | - -## K-neareast neighbours (KNN) - -The Redis command [FT.SEARCH]({{< relref "commands/ft.search" >}}) takes the index name, the query string, and additional query parameters as arguments. You need to pass the number of nearest neighbors, the vector field name, and the vector's binary representation in the following way: - -``` -FT.SEARCH index "(*)=>[KNN num_neighbours @field $vector]" PARAMS 2 vector "binary_data" DIALECT 2 -``` - -Here is a more detailed explanation of this query: - -1. **Pre-filter**: The first expression within the round brackets is a filter. It allows you to decide which vectors should be taken into account before the vector search is performed. The expression `(*)` means that all vectors are considered. -2. **Next step**: The `=>` arrow indicates that the pre-filtering happens before the vector search. -3. **KNN query**: The expression `[KNN num_neighbours @field $vector]` is a parameterized query expression. A parameter name is indicated by the `$` prefix within the query string. -4. **Vector binary data**: You need to use the `PARAMS` argument to substitute `$vector` with the binary representation of the vector. The value `2` indicates that `PARAMS` is followed by two arguments, the parameter name `vector` and the parameter value. -5. **Dialect**: The vector search feature has been available since version two of the query dialect. - -You can read more about the `PARAMS` argument in the [FT.SEARCH]({{< relref "commands/ft.search" >}}) command reference. - -The following example shows you how to query for three bikes based on their description embeddings, and by using the field alias `vector`. The result is returned in ascending order based on the distance. You can see that the query only returns the fields `__vector_score` and `description`. The field `__vector_score` is present by default. Because you can have multiple vector fields in your schema, the vector score field name depends on the name of the vector field. If you change the field name `@vector` to `@foo`, the score field name changes to `__foo_score`. - -{{< clients-example query_vector vector1 >}} -FT.SEARCH idx:bikes_vss "(*)=>[KNN 3 @vector $query_vector]" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY "__vector_score" ASC RETURN 2 "__vector_score" "description" DIALECT 2 -{{< /clients-example >}} - - - -{{% alert title="Note" color="warning" %}} -The binary value of the query vector is significantly shortened in the CLI example above. -{{% /alert %}} - - -## Radius - -Instead of the number of nearest neighbors, you need to pass the radius along with the index name, the vector field name, and the vector's binary value: - -``` -FT.SEARCH index "@field:[VECTOR_RANGE radius $vector]" PARAMS 2 vector "binary_data" DIALECT 2 -``` - -If you want to sort by distance, then you must yield the distance via the range query parameter `$YIELD_DISTANCE_AS`: - -``` -FT.SEARCH index "@field:[VECTOR_RANGE radius $vector]=>{$YIELD_DISTANCE_AS: dist_field}" PARAMS 2 vector "binary_data" SORTBY dist_field DIALECT 2 -``` - -Here is a more detailed explanation of this query: - -1. **Range query**: the syntax of a radius query is very similar to the regular range query, except for the keyword `VECTOR_RANGE`. You can also combine a vector radius query with other queries in the same way as regular range queries. See [combined queries article]({{< relref "/develop/interact/search-and-query/query/combined" >}}) for more details. -2. **Additional step**: the `=>` arrow means that the range query is followed by evaluating additional parameters. -3. **Range query parameters**: parameters such as `$YIELD_DISTANCE_AS` can be found in the [vectors reference documentation]({{< relref "/develop/interact/search-and-query/advanced-concepts/vectors" >}}). -4. **Vector binary data**: you need to use `PARAMS` to pass the binary representation of the vector. -5. **Dialect**: vector search has been available since version two of the query dialect. - - -{{% alert title="Note" color="warning" %}} -By default, [`FT.SEARCH`]({{< relref "commands/ft.search/" >}}) returns only the first ten results. The [range query article]({{< relref "/develop/interact/search-and-query/query/range" >}}) explains to you how to scroll through the result set. -{{% /alert %}} - -The example below shows a radius query that returns the description and the distance within a radius of `0.5`. The result is sorted by the distance. - -{{< clients-example query_vector vector2 >}} -FT.SEARCH idx:bikes_vss "@vector:[VECTOR_RANGE 0.5 $query_vector]=>{$YIELD_DISTANCE_AS: vector_dist}" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY vector_dist ASC RETURN 2 vector_dist description DIALECT 2 -{{< /clients-example >}} - - diff --git a/content/develop/get-started/_index.md b/content/develop/get-started/_index.md index 0bd65d2a5e..c2d1189a2e 100644 --- a/content/develop/get-started/_index.md +++ b/content/develop/get-started/_index.md @@ -22,7 +22,7 @@ Redis can be used as a database, cache, streaming engine, message broker, and mo 1. [Data structure store]({{< relref "/develop/get-started/data-store" >}}) 2. [Document database]({{< relref "/develop/get-started/document-database" >}}) -3. [Vector database]({{< relref "/develop/ai/vector-database" >}}) +3. [Vector database]({{< relref "/develop/get-started/vector-database" >}}) Please select the guide that aligns best with your specific usage scenario. diff --git a/content/develop/ai/rag.md b/content/develop/get-started/rag.md similarity index 94% rename from content/develop/ai/rag.md rename to content/develop/get-started/rag.md index dfc5e6c02b..b5d06af6ff 100644 --- a/content/develop/ai/rag.md +++ b/content/develop/get-started/rag.md @@ -9,12 +9,11 @@ categories: - oss - kubernetes - clients -description: Understand how to use Redis for Retrieval Augmented Generation -linkTitle: Overview of RAG with Redis +description: Understand how to use Redis for RAG use cases +linkTitle: RAG with Redis stack: true -title: Overview of RAG with Redis +title: RAG with Redis weight: 4 -aliases: /develop/get-started/rag --- ### What is Retrieval Augmented Generation (RAG)? Large Language Models (LLMs) generate human-like text but are limited by the data they were trained on. RAG enhances LLMs by integrating them with external, domain-specific data stored in a Redis [vector database]({{< relref "/develop/get-started/vector-database" >}}). @@ -48,7 +47,7 @@ To build a RAG application with Redis, follow these general steps: 1. **Popular AI frameworks**: Redis integrates seamlessly with various AI frameworks and tools. For instance, combining Redis with [LangChain](https://python.langchain.com/v0.2/docs/integrations/vectorstores/redis/) or [LlamaIndex](https://docs.llamaindex.ai/en/latest/examples/vector_stores/RedisIndexDemo/), libraries for building language models, enables developers to create sophisticated RAG pipelines. These integrations support efficient data management and building real-time LLM chains. 1. **Spring AI and Redis**: Using [Spring AI with Redis](https://redis.io/blog/building-a-rag-application-with-redis-and-spring-ai/) simplifies building RAG applications. Spring AI provides a structured approach to integrating AI capabilities into applications, while Redis handles data management, ensuring the RAG pipeline is efficient and scalable. -1. **Embed and store data**: Convert your data into vector embeddings using a suitable model (e.g., BERT, GPT). Store these embeddings in Redis, where they can be quickly retrieved based on vector searches. The [client library guides]({{< relref "/develop/clients" >}}) have code examples showing how to do this. +1. **Embed and store data**: Convert your data into vector embeddings using a suitable model (e.g., BERT, GPT). Store these embeddings in Redis, where they can be quickly retrieved based on vector searches. 1. **Integrate with a generative model**: Use a generative AI model that can leverage the retrieved data. The model will use the vectors stored in Redis to augment its generation process, ensuring the output is informed by relevant, up-to-date information. @@ -73,4 +72,4 @@ In summary, Redis offers a powerful and efficient platform for implementing RAG. ## Continue learning with Redis University -{{< university-links >}} +{{< university-links >}} \ No newline at end of file diff --git a/content/develop/ai/genai-apps.md b/content/develop/get-started/redis-in-ai.md similarity index 72% rename from content/develop/ai/genai-apps.md rename to content/develop/get-started/redis-in-ai.md index a8ec088fdc..9494bd62ff 100644 --- a/content/develop/ai/genai-apps.md +++ b/content/develop/get-started/redis-in-ai.md @@ -1,25 +1,29 @@ --- -aliases: /develop/get-started/redis-in-ai Title: Redis for GenAI apps alwaysopen: false categories: - docs - develop description: Understand key benefits of using Redis for AI. -linktitle: GenAI tutorials and demos +linktitle: GenAI apps weight: 20 --- -Redis supports storing and indexing vector embeddings—dense numeric representations of unstructured data like text, images, or audio. These embeddings capture semantic meaning, which makes it possible to perform similarity searches using approximate nearest neighbor (ANN) algorithms and K-Nearest Neighbor (KNN) searches. You can use Redis vector search to retrieve relevant content based on a query vector, such as similar questions or documents. This approach is useful for augmenting prompts in large language model (LLM) workflows. By retrieving only the most relevant data before calling the LLM, you can reduce inference costs and improve latency. +Redis enables high-performance, scalable, and reliable data management, making it a key component for GenAI apps, chatbots, and AI agents. By leveraging Redis for fast data retrieval, caching, and vector search capabilities, you can enhance AI-powered interactions, reduce latency, and improve user experience. + +Redis excels in storing and indexing vector embeddings that semantically represent unstructured data. With vector search, Redis retrieves similar questions and relevant data, lowering LLM inference costs and latency. It fetches pertinent portions of chat history, enriching context for more accurate and relevant responses. These features make Redis an ideal choice for RAG systems and GenAI apps requiring fast data access. ## Key Benefits of Redis in GenAI Apps - **Performance**: low-latency data access enables real-time interactions critical for AI-driven applications. -- **Scalability**: Redis is designed to handle numerous concurrent connections, so it is perfect for high-demand GenAI apps. +- **Scalability**: designed to handle numerous concurrent connections, Redis is perfect for high-demand GenAI apps. - **Caching**: efficiently stores frequently accessed data and responses, reducing primary database load and accelerating response times. - **Session Management**: in-memory data structures simplify managing session states in conversational AI scenarios. - **Flexibility**: Redis supports diverse data structures (for example, strings, hashes, lists, sets), allowing tailored solutions for GenAI apps. -- **Easy integration**: [RedisVL]({{< relref "/integrate/redisvl" >}}) is a Python library with an integrated CLI, offering seamless integration with Redis to enhance GenAI applications. + +[RedisVL]({{< relref "/integrate/redisvl" >}}) is a Python library with an integrated CLI, offering seamless integration with Redis to enhance GenAI applications. + +--- ## Redis Use Cases in GenAI Apps @@ -57,6 +61,8 @@ Redis enhances natural language understanding by: See the [Streaming LLM Output Using Redis Streams](https://redis.io/learn/howtos/solutions/streams/streaming-llm-output) tutorial for an in-depth walkthrough. +Redis is a powerful tool to elevate your GenAI applications, enabling them to deliver superior performance, scalability, and user satisfaction. + ## Resources Check out the [Redis for AI]({{< relref "/develop/ai" >}}) documentation for getting started guides, concepts, ecosystem integrations, examples, and Python notebooks. diff --git a/content/develop/ai/vector-database.md b/content/develop/get-started/vector-database.md similarity index 97% rename from content/develop/ai/vector-database.md rename to content/develop/get-started/vector-database.md index 5ea17c3dbf..fea1bc77e6 100644 --- a/content/develop/ai/vector-database.md +++ b/content/develop/get-started/vector-database.md @@ -9,12 +9,11 @@ categories: - oss - kubernetes - clients -description: Understand how to use Redis vector database -linkTitle: Vector database quick start +description: Understand how to use Redis as a vector database +linkTitle: Vector database stack: true -title: Redis vector database quick start guide -weight: 1 -aliases: /develop/get-started/vector-database +title: Redis as a vector database quick start guide +weight: 3 --- This quick start guide helps you to: @@ -59,7 +58,7 @@ The easiest way to get started is to use Redis Cloud: 1. Create a [free account](https://redis.com/try-free?utm_source=redisio&utm_medium=referral&utm_campaign=2023-09-try_free&utm_content=cu-redis_cloud_users). - {{Create your database with Essentials selected.}} + 2. Follow the instructions to create a free database. This free Redis Cloud database comes out of the box with all the Redis Community Edition features. @@ -285,4 +284,4 @@ From the description, this bike is an excellent match for younger children, and ## Continue learning with Redis University -{{< university-links >}} +{{< university-links >}} \ No newline at end of file diff --git a/content/develop/ai/vector-fields.md b/content/develop/interact/search-and-query/advanced-concepts/vectors.md similarity index 99% rename from content/develop/ai/vector-fields.md rename to content/develop/interact/search-and-query/advanced-concepts/vectors.md index 8285e0232a..b8a20e356c 100644 --- a/content/develop/ai/vector-fields.md +++ b/content/develop/interact/search-and-query/advanced-concepts/vectors.md @@ -10,11 +10,10 @@ categories: - kubernetes - clients description: Learn how to use vector fields and perform vector searches in Redis -linkTitle: Advanced vector queries guide +linkTitle: Vectors math: true -title: Perform advanced queries with vector fields -weight: 3 -aliases: /develop/interact/search-and-query/advanced-concepts/vectors +title: Vectors +weight: 14 --- Redis includes a [high-performance vector database](https://redis.io/blog/benchmarking-results-for-vector-databases/) that lets you perform semantic searches over vector embeddings. You can augment these searches with filtering over text, numerical, geospatial, and tag metadata. @@ -524,3 +523,7 @@ Here are some additonal resources that apply vector search for different use cas - [Retrieval augmented generation from scratch](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/01_redisvl.ipynb) - [Semantic caching](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/semantic_caching_gemini.ipynb) + +## Continue learning with Redis University + +{{< university-links >}} diff --git a/content/develop/interact/search-and-query/query/vector-search.md b/content/develop/interact/search-and-query/query/vector-search.md index 3bb226a99d..4178cd8c88 100644 --- a/content/develop/interact/search-and-query/query/vector-search.md +++ b/content/develop/interact/search-and-query/query/vector-search.md @@ -15,5 +15,93 @@ title: Vector search weight: 5 --- -See the [Redis vector database quick start guide]({{< relref "/develop/ai/vector-database" >}}) for more information about getting started. You can also find examples in the [vector search examples page]({{< relref "/develop/ai/vector-search" >}}) and detailed information about how to [perform advanced queries with vector fields]({{< relref "/develop/ai/vector-fields" >}}). +This article gives you a good overview of how to perform vector search queries with the Redis Query Engine, which is part of Redis Community Edition. See the [Redis as a vector database quick start guide]({{< relref "/develop/get-started/vector-database" >}}) for more information about Redis as a vector database. You can also find more detailed information about all the parameters in the [vector reference documentation]({{< relref "/develop/interact/search-and-query/advanced-concepts/vectors" >}}). +A vector search query on a vector field allows you to find all vectors in a vector space that are close to a given vector. You can query for the k-nearest neighbors or vectors within a given radius. + +The examples in this article use a schema with the following fields: + +| JSON field | Field alias | Field type | Description | +| ------------------------ | ----------- | ----------- | ----------- | +| `$.description` | `description` | `TEXT` | The description of a bicycle as unstructured text | +| `$.description_embeddings` | `vector` | `VECTOR` | The vector that a machine learning model derived from the description text | + +## K-neareast neighbours (KNN) + +The Redis command [FT.SEARCH]({{< relref "commands/ft.search" >}}) takes the index name, the query string, and additional query parameters as arguments. You need to pass the number of nearest neighbors, the vector field name, and the vector's binary representation in the following way: + +``` +FT.SEARCH index "(*)=>[KNN num_neighbours @field $vector]" PARAMS 2 vector "binary_data" DIALECT 2 +``` + +Here is a more detailed explanation of this query: + +1. **Pre-filter**: The first expression within the round brackets is a filter. It allows you to decide which vectors should be taken into account before the vector search is performed. The expression `(*)` means that all vectors are considered. +2. **Next step**: The `=>` arrow indicates that the pre-filtering happens before the vector search. +3. **KNN query**: The expression `[KNN num_neighbours @field $vector]` is a parameterized query expression. A parameter name is indicated by the `$` prefix within the query string. +4. **Vector binary data**: You need to use the `PARAMS` argument to substitute `$vector` with the binary representation of the vector. The value `2` indicates that `PARAMS` is followed by two arguments, the parameter name `vector` and the parameter value. +5. **Dialect**: The vector search feature has been available since version two of the query dialect. + +You can read more about the `PARAMS` argument in the [FT.SEARCH]({{< relref "commands/ft.search" >}}) command reference. + +The following example shows you how to query for three bikes based on their description embeddings, and by using the field alias `vector`. The result is returned in ascending order based on the distance. You can see that the query only returns the fields `__vector_score` and `description`. The field `__vector_score` is present by default. Because you can have multiple vector fields in your schema, the vector score field name depends on the name of the vector field. If you change the field name `@vector` to `@foo`, the score field name changes to `__foo_score`. + +{{< clients-example query_vector vector1 >}} +FT.SEARCH idx:bikes_vss "(*)=>[KNN 3 @vector $query_vector]" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY "__vector_score" ASC RETURN 2 "__vector_score" "description" DIALECT 2 +{{< /clients-example >}} + + + +{{% alert title="Note" color="warning" %}} +The binary value of the query vector is significantly shortened in the CLI example above. +{{% /alert %}} + + +## Radius + +Instead of the number of nearest neighbors, you need to pass the radius along with the index name, the vector field name, and the vector's binary value: + +``` +FT.SEARCH index "@field:[VECTOR_RANGE radius $vector]" PARAMS 2 vector "binary_data" DIALECT 2 +``` + +If you want to sort by distance, then you must yield the distance via the range query parameter `$YIELD_DISTANCE_AS`: + +``` +FT.SEARCH index "@field:[VECTOR_RANGE radius $vector]=>{$YIELD_DISTANCE_AS: dist_field}" PARAMS 2 vector "binary_data" SORTBY dist_field DIALECT 2 +``` + +Here is a more detailed explanation of this query: + +1. **Range query**: the syntax of a radius query is very similar to the regular range query, except for the keyword `VECTOR_RANGE`. You can also combine a vector radius query with other queries in the same way as regular range queries. See [combined queries article]({{< relref "/develop/interact/search-and-query/query/combined" >}}) for more details. +2. **Additional step**: the `=>` arrow means that the range query is followed by evaluating additional parameters. +3. **Range query parameters**: parameters such as `$YIELD_DISTANCE_AS` can be found in the [vectors reference documentation]({{< relref "/develop/interact/search-and-query/advanced-concepts/vectors" >}}). +4. **Vector binary data**: you need to use `PARAMS` to pass the binary representation of the vector. +5. **Dialect**: vector search has been available since version two of the query dialect. + + +{{% alert title="Note" color="warning" %}} +By default, [`FT.SEARCH`]({{< relref "commands/ft.search/" >}}) returns only the first ten results. The [range query article]({{< relref "/develop/interact/search-and-query/query/range" >}}) explains to you how to scroll through the result set. +{{% /alert %}} + +The example below shows a radius query that returns the description and the distance within a radius of `0.5`. The result is sorted by the distance. + +{{< clients-example query_vector vector2 >}} +FT.SEARCH idx:bikes_vss "@vector:[VECTOR_RANGE 0.5 $query_vector]=>{$YIELD_DISTANCE_AS: vector_dist}" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY vector_dist ASC RETURN 2 vector_dist description DIALECT 2 +{{< /clients-example >}} + + \ No newline at end of file diff --git a/static/resources/redisuniversity.yaml b/static/resources/redisuniversity.yaml index 86062a76ca..d2ec3d78f4 100644 --- a/static/resources/redisuniversity.yaml +++ b/static/resources/redisuniversity.yaml @@ -6,6 +6,15 @@ links: - page: /develop/get-started/document-database text: See the [Get Started with Redis learning path](https://university.redis.io/learningpath/14q8m6gilfwltm) for courses. + - page: /develop/get-started/rag + text: See the [Vector Advanced Topics course](https://university.redis.io/course/i3fv2hbhqnpni8) to learn more. + + - page: /develop/get-started/redis-in-ai + text: See the [Vector Advanced Topics course](https://university.redis.io/course/i3fv2hbhqnpni8) to learn more. + + - page: /develop/get-started/vector-database + text: See the [Redis vector database course](https://university.redis.io/course/7e2qbbeg963twz) for learning modules. + # Redis Cloud links - page: /operate/rc/databases/monitor-performance text: See the [Redis Cloud monitoring overview course](https://university.redis.io/course/sdm9uvjmd0iedp) to learn more. @@ -57,11 +66,3 @@ links: - page: /develop/interact/search-and-query/advanced-concepts/vectors text: See the [Introduction to vector search course](https://university.redis.io/course/yz1lretjfpdlew?tab=details) to learn more. - - page: /develop/ai/rag - text: See the [Vector Advanced Topics course](https://university.redis.io/course/i3fv2hbhqnpni8) to learn more. - - - page: /develop/ai/genai-apps - text: See the [Vector Advanced Topics course](https://university.redis.io/course/i3fv2hbhqnpni8) to learn more. - - - page: /develop/ai/vector-database - text: See the [Redis vector database course](https://university.redis.io/course/7e2qbbeg963twz) for learning modules.