From ea7ec972b2e7eaa164166ec6c24b4ba1d48c898a Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Wed, 1 Oct 2025 18:46:47 +0200 Subject: [PATCH 1/6] [DOCS] Clarify Update API does not replace fields with sparse_vector field MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updated the documentation for `sparse_vector` fields to clarify merging behavior, replacement methods, and limitations. 🚗 Compiled all the admonitions into a limitations section --- .../mapping-reference/sparse-vector.md | 79 ++++++++++++++++--- 1 file changed, 67 insertions(+), 12 deletions(-) diff --git a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md index 5cb009ddcf302..3ad97f97b58d9 100644 --- a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md +++ b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md @@ -220,24 +220,79 @@ GET my-index-000001/_search } ``` -::::{note} -`sparse_vector` fields can not be included in indices that were **created** on {{es}} versions between 8.0 and 8.10 -:::: +## Updating `sparse_vector` fields +When using the [Update API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-update) with the `doc` parameter, `sparse_vector` fields behave like nested objects and are **merged** rather than replaced. This means: -::::{note} -`sparse_vector` fields only support strictly positive values. Negative values will be rejected. -:::: +- Existing tokens in the sparse vector are preserved +- New tokens are added +- Tokens present in both the existing and new data will have their values updated +This is different from primitive array fields (like `keyword`), which are replaced entirely during updates. -::::{note} -`sparse_vector` fields do not support [analyzers](docs-content://manage-data/data-store/text-analysis.md), querying, sorting or aggregating. They may only be used within specialized queries. The recommended query to use on these fields are [`sparse_vector`](/reference/query-languages/query-dsl/query-dsl-sparse-vector-query.md) queries. They may also be used within legacy [`text_expansion`](/reference/query-languages/query-dsl/query-dsl-text-expansion-query.md) queries. -:::: +### Example of merging behavior +Original document: -::::{note} -`sparse_vector` fields only preserve 9 significant bits for the precision, which translates to a relative error of about 0.4%. -:::: +```console +PUT /my-index/_doc/1 +{ + "my_vector": { + "token_a": 0.5, + "token_b": 0.8 + } +} +``` + +Partial update: + +```console +POST /my-index/_update/1 +{ + "doc": { + "my_vector": { + "token_c": 0.3 + } + } +} +``` + +Observe that tokens are merged, not replaced: + +```json +{ + "my_vector": { + "token_a": 0.5, + "token_b": 0.8, + "token_c": 0.3 + } +} +``` + +### Replacing the entire `sparse_vector` field + +To replace the entire contents of a `sparse_vector` field, use a [script](docs-content://explore-analyze/scripting/modules-scripting-using.md) in your update request: + +```console +POST /my-index/_update/1 +{ + "script": { + "source": "ctx._source.my_vector = params.new_vector", + "params": { + "new_vector": { + "token_x": 1.0, + "token_y": 0.6 + } + } + } +} +``` +This same merging behavior also applies to [`rank_features` fields](/mapping-reference/rank-feature.md), as they are also object-like structures. +## Important notes and limitations +- `sparse_vector` fields can not be included in indices that were **created** on Elasticsearch versions between 8.0 and 8.10 +- `sparse_vector` fields only support strictly positive values. Negative values will be rejected. +- `sparse_vector` fields do not support [analyzers](https://www.elastic.co/docs/manage-data/data-store/text-analysis), querying, sorting or aggregating. They may only be used within specialized queries. The recommended query to use on these fields are [`sparse_vector`](https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-sparse-vector-query) queries. They may also be used within legacy [`text_expansion`](https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-text-expansion-query) queries. +- `sparse_vector` fields only preserve 9 significant bits for the precision, which translates to a relative error of about 0.4%. From 7ff0596149f34b35b88db2e1af007a58a09b6060 Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Wed, 1 Oct 2025 18:51:12 +0200 Subject: [PATCH 2/6] Fix links --- .../elasticsearch/mapping-reference/sparse-vector.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md index 3ad97f97b58d9..ef2728dc57bbc 100644 --- a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md +++ b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md @@ -288,11 +288,11 @@ POST /my-index/_update/1 } ``` -This same merging behavior also applies to [`rank_features` fields](/mapping-reference/rank-feature.md), as they are also object-like structures. +This same merging behavior also applies to [`rank_features` fields](/reference/mapping-reference/rank-feature.md), as they are also object-like structures. ## Important notes and limitations -- `sparse_vector` fields can not be included in indices that were **created** on Elasticsearch versions between 8.0 and 8.10 +- `sparse_vector` fields can not be included in indices that were **created** on {{es}} versions between 8.0 and 8.10 - `sparse_vector` fields only support strictly positive values. Negative values will be rejected. -- `sparse_vector` fields do not support [analyzers](https://www.elastic.co/docs/manage-data/data-store/text-analysis), querying, sorting or aggregating. They may only be used within specialized queries. The recommended query to use on these fields are [`sparse_vector`](https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-sparse-vector-query) queries. They may also be used within legacy [`text_expansion`](https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-text-expansion-query) queries. +- `sparse_vector` fields do not support [analyzers](docs-content://manage-data/data-store/text-analysis.md), querying, sorting or aggregating. They may only be used within specialized queries. The recommended query to use on these fields are [`sparse_vector`](/reference/query-languages/query-dsl/query-dsl-sparse-vector-query.md) queries. They may also be used within legacy [`text_expansion`](/reference/query-languages/query-dsl/query-dsl-text-expansion-query.md) queries. - `sparse_vector` fields only preserve 9 significant bits for the precision, which translates to a relative error of about 0.4%. From 75ac09a81d3049ec10704ece889fda9a12b4f9bc Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Wed, 1 Oct 2025 18:54:38 +0200 Subject: [PATCH 3/6] Fix rank link to rank_features --- docs/reference/elasticsearch/mapping-reference/sparse-vector.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md index ef2728dc57bbc..6397205be1f16 100644 --- a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md +++ b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md @@ -288,7 +288,7 @@ POST /my-index/_update/1 } ``` -This same merging behavior also applies to [`rank_features` fields](/reference/mapping-reference/rank-feature.md), as they are also object-like structures. +This same merging behavior also applies to [`rank_features` fields](/reference/mapping-reference/rank-features.md), as they are also object-like structures. ## Important notes and limitations From b89b08f9a099766e3545e3a2e32795c213bf5bef Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Wed, 1 Oct 2025 18:57:09 +0200 Subject: [PATCH 4/6] idem --- docs/reference/elasticsearch/mapping-reference/sparse-vector.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md index 6397205be1f16..c722a423c2eb5 100644 --- a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md +++ b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md @@ -288,7 +288,7 @@ POST /my-index/_update/1 } ``` -This same merging behavior also applies to [`rank_features` fields](/reference/mapping-reference/rank-features.md), as they are also object-like structures. +This same merging behavior also applies to [`rank_features` fields](/reference/elasticsearch/mapping-reference/rank-features.md), as they are also object-like structures. ## Important notes and limitations From accff8b1dab6f6a319e8176090fdaa55ac704ef5 Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Wed, 1 Oct 2025 18:59:25 +0200 Subject: [PATCH 5/6] Fix formatting and wording --- .../elasticsearch/mapping-reference/sparse-vector.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md index c722a423c2eb5..34dd11b42938b 100644 --- a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md +++ b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md @@ -288,11 +288,13 @@ POST /my-index/_update/1 } ``` -This same merging behavior also applies to [`rank_features` fields](/reference/elasticsearch/mapping-reference/rank-features.md), as they are also object-like structures. +:::{note} +This same merging behavior also applies to [`rank_features` fields](/reference/elasticsearch/mapping-reference/rank-features.md), because they are also object-like structures. +::: ## Important notes and limitations -- `sparse_vector` fields can not be included in indices that were **created** on {{es}} versions between 8.0 and 8.10 +- `sparse_vector` fields cannot be included in indices that were **created** on {{es}} versions between 8.0 and 8.10 - `sparse_vector` fields only support strictly positive values. Negative values will be rejected. - `sparse_vector` fields do not support [analyzers](docs-content://manage-data/data-store/text-analysis.md), querying, sorting or aggregating. They may only be used within specialized queries. The recommended query to use on these fields are [`sparse_vector`](/reference/query-languages/query-dsl/query-dsl-sparse-vector-query.md) queries. They may also be used within legacy [`text_expansion`](/reference/query-languages/query-dsl/query-dsl-text-expansion-query.md) queries. - `sparse_vector` fields only preserve 9 significant bits for the precision, which translates to a relative error of about 0.4%. From e588b388709a562969469647138a11540b66a73b Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Wed, 1 Oct 2025 19:02:46 +0200 Subject: [PATCH 6/6] Add applies_to and products metadata --- .../elasticsearch/mapping-reference/sparse-vector.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md index 34dd11b42938b..d7cdee6c8f218 100644 --- a/docs/reference/elasticsearch/mapping-reference/sparse-vector.md +++ b/docs/reference/elasticsearch/mapping-reference/sparse-vector.md @@ -1,4 +1,9 @@ --- +applies_to: + stack: + serverless: +products: + - id: elasticsearch navigation_title: "Sparse vector" mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/sparse-vector.html