add rough draft for multimodal search guide

Author: guimachiavelli

.code-samples.meilisearch.yaml

@@ -1409,7 +1409,9 @@ search_parameter_guide_vector_1: |-
     -H 'content-type: application/json' \
     --data-binary '{ 
       "vector": [0, 1, 2],
-      "embedder": "EMBEDDER_NAME"
+      "hybrid": {
+        "embedder": "EMBEDDER_NAME"
+      }
     }'
 search_parameter_reference_retrieve_vectors_1: |-
   curl -X POST 'MEILISEARCH_URL/indexes/INDEX_NAME/search' \

config/sidebar-learn.json

@@ -54,6 +54,11 @@
         "label": "Which embedder should I choose?",
         "slug": "choose_an_embedder"
       },
+      {
+        "source": "learn/ai_powered_search/multimodal.mdx",
+        "label": "Image search with user-provided embeddings",
+        "slug": "multimodal_search"
+      },
       {
         "source": "learn/ai_powered_search/difference_full_text_ai_search.mdx",
         "label": "Differences between full-text and AI-powered search",

learn/ai_powered_search/multimodal.mdx

@@ -1,17 +1,30 @@
 ---
-title: Implement image search — Meilisearch documentation
+title: Image search with user-provided embeddings — Meilisearch documentation
 description: 
 ---
 
-# Use AI-powered search with user-provided embeddings
+# Image search with user-provided embeddings
 
-This guide shows how to perform AI-powered searches with user-generated embeddings instead of relying on a third-party tool.
+This articles shows the main steps for performing multimodal searches where users use text to search through a database of images with no associated metadata.
 
 ## Requirements
 
-- A Meilisearch project with AI-powered search activated
+- A database of images
+- A Meilisearch project
 
-## Configure a custom embedder
+## Configure your local embedding generation pipeline
+
+First, set up a system that sends your images to your chosen embedding generation tool.
+
+The exact procedure for this depend heavily on your specific setup, but should include these main steps:
+
+1. Choose a provider you can run locally
+2. Choose a model that supports both image and text input
+3. Send your images to the provider
+4. Add the returned embeddings to the `_vector` field for each image in your database
+5. Periodically send your vectorized documents to Meilisearch
+
+## Configure a user-provided embedder
 
 Configure the `embedder` index setting, settings its source to `userProvided`:
 
@@ -21,49 +34,66 @@ curl \
   -H 'Content-Type: application/json' \
   --data-binary '{
     "embedders": {
-      "image2text": {
+      "EMBEDDER_NAME": {
         "source":  "userProvided",
-        "dimensions": 3
+        "dimensions": MODEL_DIMENSIONS
       }
     }
   }'
 ```
 
+Replace `EMBEDDER_NAME` with the name you wish to give your embedder. Replace `MODEL_DIMENSIONS` with the number of dimensions of your chosen model.
+
 ## Add documents to Meilisearch
 
-Next, use [the `/documents` endpoint](/reference/api/documents?utm_campaign=vector-search&utm_source=docs&utm_medium=vector-search-guide) to upload vectorized documents. Place vector data in your documents' `_vectors` field:
+Next, use [the `/documents` endpoint](/reference/api/documents) to upload the vectorized images.
 
-```sh
-curl -X POST -H 'content-type: application/json' \
-'localhost:7700/indexes/products/documents' \
---data-binary '[
-    { "id": 0, "_vectors": {"image2text": [0, 0.8, -0.2]}, "text": "frying pan" },
-    { "id": 1, "_vectors": {"image2text": [1, -0.2, 0]}, "text": "baking dish" }
-]'
-```
+In most cases, you should automate this step so Meilisearch is up to date with your primary database.
 
-## Vector search with user-provided embeddings
+## Set up pipeline for vectorizing queries
 
-When using a custom embedder, you must vectorize both your documents and user queries.
+Since you are using a `userProvided` embedder, you must also generate the embeddings for the search query. This process should be similar to generating embeddings for your images:
 
-Once you have the query's vector, pass it to the `vector` search parameter to perform an AI-powered search:
+1. Receive user query from your front-end
+2. Vectorize query with your local embedding provider
+3. Perform search using the vectorized query
+
+## Vector search with user-provided embeddings
+
+Once you have the query's vector, pass it to the `vector` search parameter to perform a  semantic AI-powered search:
 
 ```sh
 curl -X POST -H 'content-type: application/json' \
   'localhost:7700/indexes/products/search' \
-  --data-binary '{ "vector": [0, 1, 2] }'
+  --data-binary '{ 
+    "vector": VECTORIZED_QUERY,
+    "hybrid": {
+      "embedder": "EMBEDDER_NAME",
+    }
+  }'
 ```
 
-`vector` must be an array of numbers indicating the search vector. You must generate these yourself when using vector search with user-provided embeddings.
+Replace `VECTORIZED_QUERY` with the embedding generated by your provider and `EMBEDDER_NAME` with your embedder.
 
-`vector` can be used together with [other search parameters](/reference/api/search?utm_campaign=vector-search&utm_source=docs&utm_medium=vector-search-guide), including [`filter`](/reference/api/search#filter) and [`sort`](/reference/api/search#sort):
+If your images have any associated metadata, you may perform a hybrid search by including the original `q`:
 
 ```sh
 curl -X POST -H 'content-type: application/json' \
   'localhost:7700/indexes/products/search' \
-  --data-binary '{
-    "vector": [0, 1, 2],
-    "filter": "price < 10",
-    "sort": ["price:asc"]
+  --data-binary '{ 
+    "vector": VECTORIZED_QUERY,
+    "hybrid": {
+      "embedder": "EMBEDDER_NAME",
+    }
+    "q": "QUERY",
   }'
 ```
+
+## Conclusion
+
+You have seen the main steps for implementing multimodal search with Meilisearch:
+
+1. Prepare a pipeline that converts your images into vectors
+2. Index the vectorized images with Meilisearch
+3. Prepare a pipeline that converts your users' queries into vectors
+4. Perform searches using the converted queries