From ffcbcf3bb6b57e11caf9aa5ad0442c6418561dc9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Wed, 27 Aug 2025 18:24:47 +0200 Subject: [PATCH 1/9] Adds recommendation about choosing project profile. --- .../search/serverless-elasticsearch-get-started.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/solutions/search/serverless-elasticsearch-get-started.md b/solutions/search/serverless-elasticsearch-get-started.md index 8f68135775..c3c4c42bfb 100644 --- a/solutions/search/serverless-elasticsearch-get-started.md +++ b/solutions/search/serverless-elasticsearch-get-started.md @@ -55,14 +55,24 @@ Use your {{ecloud}} account to create a fully-managed {{es}} project: 6. Once the project is ready, select **Continue**. ::::{tip} + +**The General Purpose (GP) profile is recommended when your use case is based on:** + +- full text search +- sparse vector-based semantic search (such as ELSER) +- dense vectors compressed using BBQ (enabled by default) +- sparse vectors + +**The Vector Optimized (VO) profile is recommended when your use case is based on:** + +- uncompressed dense vectors with high dimensionality + Learn how billing works for your project in [Elasticsearch billing dimensions](../../deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md). :::: - Now your project is ready to start creating indices, adding data, and performing searches. You can choose one of the following options to proceed. - ## Option 1: Follow the guided index flow [elasticsearch-follow-guided-index-flow] Once your project is set up, you’ll be directed to a page where you can create your first index. An index is where documents are stored and organized, making it possible to search and retrieve data. From e1df57db47e006d9e3425d7567f3b90a670751aa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Thu, 28 Aug 2025 09:20:47 +0200 Subject: [PATCH 2/9] Addresses feedback. --- solutions/search/serverless-elasticsearch-get-started.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/solutions/search/serverless-elasticsearch-get-started.md b/solutions/search/serverless-elasticsearch-get-started.md index c3c4c42bfb..9c8949740b 100644 --- a/solutions/search/serverless-elasticsearch-get-started.md +++ b/solutions/search/serverless-elasticsearch-get-started.md @@ -56,10 +56,10 @@ Use your {{ecloud}} account to create a fully-managed {{es}} project: ::::{tip} -**The General Purpose (GP) profile is recommended when your use case is based on:** +**The General Purpose (GP) profile is recommended for most search use cases such as:** - full text search -- sparse vector-based semantic search (such as ELSER) +- sparse vector-based semantic search (such as when using ELSER) - dense vectors compressed using BBQ (enabled by default) - sparse vectors From 604ab718ed3b3679b5f2b2bdb13de1b249560f8b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Thu, 28 Aug 2025 10:31:18 +0200 Subject: [PATCH 3/9] Adds section about project profile in vector search page. --- solutions/search/vector.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/solutions/search/vector.md b/solutions/search/vector.md index de65d40a76..7b632cb484 100644 --- a/solutions/search/vector.md +++ b/solutions/search/vector.md @@ -46,3 +46,28 @@ The sparse vector approach uses the ELSER model to expand content with semantica - Large-scale deployments [Learn more about sparse vector search with ELSER](vector/sparse-vector.md). + +## Choosing your project's profile +```{applies_to} +serverless +``` + +When selecting a profile configuration for your project, it's recommended to consider your vector search use case. +You can choose between two profiles: the General Purpose profile and the Vector Optimized profile. + +**The General Purpose (GP) profile is recommended for most search use cases, including:** + +- full text search +- sparse vector-based semantic search (such as when using ELSER) +- dense vectors compressed using BBQ (enabled by default) +- sparse vectors + +**The Vector Optimized (VO) profile is recommended when your use case is based on:** + +- uncompressed dense vectors with high dimensionality + +::::{tip} + +Learn how billing works for your project in [Elasticsearch billing dimensions](../../deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md). + +:::: From b0d63b96c85a676454f851dabe23eabe42edd527 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Thu, 28 Aug 2025 10:47:07 +0200 Subject: [PATCH 4/9] Fixes format. --- solutions/search/vector.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/search/vector.md b/solutions/search/vector.md index 7b632cb484..565c46662f 100644 --- a/solutions/search/vector.md +++ b/solutions/search/vector.md @@ -49,7 +49,7 @@ The sparse vector approach uses the ELSER model to expand content with semantica ## Choosing your project's profile ```{applies_to} -serverless +serverless: ``` When selecting a profile configuration for your project, it's recommended to consider your vector search use case. From ea4b76d285e54a85aefd72c550c810160f0de98f Mon Sep 17 00:00:00 2001 From: lcawl Date: Fri, 5 Sep 2025 14:19:40 -0700 Subject: [PATCH 5/9] Revert changes to serverless-elasticsearch-get-started.md --- .../search/serverless-elasticsearch-get-started.md | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/solutions/search/serverless-elasticsearch-get-started.md b/solutions/search/serverless-elasticsearch-get-started.md index 9c8949740b..8f68135775 100644 --- a/solutions/search/serverless-elasticsearch-get-started.md +++ b/solutions/search/serverless-elasticsearch-get-started.md @@ -55,24 +55,14 @@ Use your {{ecloud}} account to create a fully-managed {{es}} project: 6. Once the project is ready, select **Continue**. ::::{tip} - -**The General Purpose (GP) profile is recommended for most search use cases such as:** - -- full text search -- sparse vector-based semantic search (such as when using ELSER) -- dense vectors compressed using BBQ (enabled by default) -- sparse vectors - -**The Vector Optimized (VO) profile is recommended when your use case is based on:** - -- uncompressed dense vectors with high dimensionality - Learn how billing works for your project in [Elasticsearch billing dimensions](../../deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md). :::: + Now your project is ready to start creating indices, adding data, and performing searches. You can choose one of the following options to proceed. + ## Option 1: Follow the guided index flow [elasticsearch-follow-guided-index-flow] Once your project is set up, you’ll be directed to a page where you can create your first index. An index is where documents are stored and organized, making it possible to search and retrieve data. From 3cf3a1a5c32ac51d0a53ee2ab2c37119f2ba98c7 Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 11 Sep 2025 17:54:35 -0700 Subject: [PATCH 6/9] Move vector profile tips --- solutions/search/vector.md | 25 -------------------- solutions/search/vector/bring-own-vectors.md | 6 +++-- solutions/search/vector/dense-vector.md | 11 +++++++++ solutions/search/vector/knn.md | 8 +++++-- 4 files changed, 21 insertions(+), 29 deletions(-) diff --git a/solutions/search/vector.md b/solutions/search/vector.md index 565c46662f..de65d40a76 100644 --- a/solutions/search/vector.md +++ b/solutions/search/vector.md @@ -46,28 +46,3 @@ The sparse vector approach uses the ELSER model to expand content with semantica - Large-scale deployments [Learn more about sparse vector search with ELSER](vector/sparse-vector.md). - -## Choosing your project's profile -```{applies_to} -serverless: -``` - -When selecting a profile configuration for your project, it's recommended to consider your vector search use case. -You can choose between two profiles: the General Purpose profile and the Vector Optimized profile. - -**The General Purpose (GP) profile is recommended for most search use cases, including:** - -- full text search -- sparse vector-based semantic search (such as when using ELSER) -- dense vectors compressed using BBQ (enabled by default) -- sparse vectors - -**The Vector Optimized (VO) profile is recommended when your use case is based on:** - -- uncompressed dense vectors with high dimensionality - -::::{tip} - -Learn how billing works for your project in [Elasticsearch billing dimensions](../../deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md). - -:::: diff --git a/solutions/search/vector/bring-own-vectors.md b/solutions/search/vector/bring-own-vectors.md index e648e08638..2e165e3379 100644 --- a/solutions/search/vector/bring-own-vectors.md +++ b/solutions/search/vector/bring-own-vectors.md @@ -20,11 +20,13 @@ You'll also learn the syntax for searching these documents using a [k-nearest ne ## Prerequisites -- If you're using {{es-serverless}}, create a project that is optimized for vectors. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. +- If you're using {{es-serverless}}, create a general purpose project. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. - If you're using {{ech}} or a self-managed cluster, start {{es}} and {{kib}}. The simplest method to complete the steps in this guide is to log in with a user that has the `superuser` built-in role. To learn about role-based access control, check out [](/deploy-manage/users-roles/cluster-or-deployment-auth/user-roles.md). +To learn about {{es-serverless}} project profiles, refer to [](dense-vector.md#vector-profiles). + ## Create a vector database When you create vectors (or _vectorize_ your data), you convert complex and nuanced content (such as text, videos, images, or audio) into multidimensional numerical representations. @@ -47,7 +49,7 @@ Each document in our simple data set will have: * An embedding of that review: stored in a `review_vector` field, which is defined as a [`dense_vector`](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md) data type. :::{tip} -The `dense_vector` type automatically uses `int8_hnsw` quantization by default to reduce the memory footprint required when searching float vectors. Learn more about balancing performance and accuracy in [Dense vector quantization](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md#dense-vector-quantization). +The `dense_vector` type automatically uses quantization by default to reduce the memory footprint required when searching float vectors. Learn more about the default quantization strategy and balancing performance and accuracy in [Dense vector field type](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md). ::: The following API request defines the `review_text` and `review_vector` fields: diff --git a/solutions/search/vector/dense-vector.md b/solutions/search/vector/dense-vector.md index 5d59e14ef0..57f9a6a1c2 100644 --- a/solutions/search/vector/dense-vector.md +++ b/solutions/search/vector/dense-vector.md @@ -27,6 +27,17 @@ Dense vector search requires both index configuration and a strategy for generat - Use the `dense_vector` field type 2. Query the index using the [`knn` search](knn.md) +### General and vector optimized projects [vector-profiles] +```{applies_to} +serverless: +``` + +When you use the [API]({{cloud-serverless-apis}}operation/operation-createelasticsearchproject) to create an {{es-serverless}} project, you can choose between two profiles: `general_purpose` and `vector`. +The general purpose profile is recommended for most search use cases, including full text search, sparse vectors, and dense vectors that use compression such as BBQ. +The vector profile is recommended only when your use case is based on uncompressed dense vectors with high dimensionality. + +For more information about how the profile affects virtual compute unit (VCU) allocation and costs, refer to [](/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md). + ## Better Binary Quantization (BBQ) [bbq] Better Binary Quantization (BBQ) is a vector quantization method for `dense_vector` fields that compresses vectors for faster and more memory-efficient similarity search. BBQ can improve relevance and cost efficiency, especially when used with HNSW. diff --git a/solutions/search/vector/knn.md b/solutions/search/vector/knn.md index ca3c5cc498..64d22d06ea 100644 --- a/solutions/search/vector/knn.md +++ b/solutions/search/vector/knn.md @@ -12,7 +12,6 @@ products: # kNN search [knn-search] - A *k-nearest neighbor* (kNN) search finds the *k* nearest vectors to a query vector, as measured by a similarity metric. Common use cases for kNN include: @@ -35,7 +34,7 @@ Common use cases for kNN include: * To run a kNN search, your data must be transformed into vectors. You can [use an NLP model in {{es}}](../../../explore-analyze/machine-learning/nlp/ml-nlp-text-emb-vector-search-example.md), or generate them outside {{es}}. - Dense vectors need to use the [`dense_vector`](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md) field type. - Queries are represented as vectors with the same dimension. You should use the same model to generate the query vector as you used to generate the document vectors. - - If you already have vectors, refer to the [Bring your own dense vectors](bring-own-vectors.md) guide. + - If you already have vectors, refer to [](bring-own-vectors.md). * To complete the steps in this guide, you must have the following [index privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md#privileges-list-indices): @@ -43,6 +42,11 @@ Common use cases for kNN include: * `create`, `index`, or `write` to add data to the index you created * `read` to search the index +:::{tip} +The default type of {{es-serverless}} project is suitable for this use case unless you plan to use uncompressed dense vectors (`int4` or `int8` quantization strategies) with high dimensionality. +Refer to [](/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md). +::: + ## kNN methods [knn-methods] {{es}} supports two methods for kNN search: From 7d6a3858a5131f5b1dacdcf09fa822ef999c7032 Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Thu, 11 Sep 2025 18:00:51 -0700 Subject: [PATCH 7/9] Update solutions/search/vector/bring-own-vectors.md --- solutions/search/vector/bring-own-vectors.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/solutions/search/vector/bring-own-vectors.md b/solutions/search/vector/bring-own-vectors.md index 7dd306fb48..3f4d88d261 100644 --- a/solutions/search/vector/bring-own-vectors.md +++ b/solutions/search/vector/bring-own-vectors.md @@ -45,7 +45,8 @@ Each document in our simple data set will have: * An embedding of that review: stored in a `review_vector` field, which is defined as a [`dense_vector`](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md) data type. :::{tip} -The `dense_vector` type automatically uses quantization by default to reduce the memory footprint required when searching float vectors. Learn more about the default quantization strategy and balancing performance and accuracy in [Dense vector field type](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md). +The `dense_vector` type automatically uses quantization by default to reduce the memory footprint when searching float vectors. +Learn more about the default quantization strategy and balancing performance and accuracy in [Dense vector field type](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md). ::: The following API request defines the `review_text` and `review_vector` fields: From 9f29541854b9b5012859d57aceb1d73a7306de3e Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 11 Sep 2025 18:04:25 -0700 Subject: [PATCH 8/9] Update link target --- solutions/search/vector/knn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/search/vector/knn.md b/solutions/search/vector/knn.md index a86f854775..1458fd5f03 100644 --- a/solutions/search/vector/knn.md +++ b/solutions/search/vector/knn.md @@ -46,7 +46,7 @@ To run a kNN search in {{es}}: :::{tip} The default type of {{es-serverless}} project is suitable for this use case unless you plan to use uncompressed dense vectors (`int4` or `int8` quantization strategies) with high dimensionality. -Refer to [](/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md). +Refer to [](dense-vector.md#vector-profiles). ::: ## kNN search methods: approxiamte and exact kNN [knn-methods] From 26143bf6320a38e7a3c84b31c7e8bbb5939fcae8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Wed, 17 Sep 2025 14:50:55 +0200 Subject: [PATCH 9/9] Apply suggestions from code review --- solutions/search/vector/dense-vector.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/search/vector/dense-vector.md b/solutions/search/vector/dense-vector.md index a996623ba2..75614442f9 100644 --- a/solutions/search/vector/dense-vector.md +++ b/solutions/search/vector/dense-vector.md @@ -30,7 +30,7 @@ To implement dense vector search in {{es}}, you need both an index configuration 2. **Query the index with k-NN search** - Use the [`knn` query](knn.md) to retrieve results based on vector similarity -### General and vector optimized projects [vector-profiles] +### General purpose and vector optimized projects [vector-profiles] ```{applies_to} serverless: ```