Update API does not replace fields with sparse_vector field

[spinscale]

Elasticsearch Version

9.1.4

Installed Plugins

No response

Java Version

bundled

OS Version

osx

Problem Description

When using the update API on a sparse_vector field, the existing contents of that field do not get removed, but the new data simply gets appended. This behaviour seems different from other fields, but makes sense from a JSON perspective.

I suppose the same issue is with rank_features, but the question is, if this is a bug or a feature.

Steps to Reproduce

DELETE alexr-test

PUT alexr-test
{
  "mappings": {
    "properties": {
      "sparse_vector": {
        "type": "sparse_vector"
      },
      "keyword" : {
        "type" : "keyword"
      }
    }
  }
}

PUT alexr-test/_doc/1?refresh
{
  "sparse_vector" : {
    "a" : 1,
    "b" : 2
  },
  "keyword": ["a", "b"]
}

POST alexr-test/_update/1?refresh
{
  "doc": {
    "sparse_vector" : {
      "c" : 3
    },
    "keyword" : ["c"]
  }
}

# this contains sparse_vector: { a:1, b:2, c:3 }
GET alexr-test/_doc/1

A workaround could be to use scripting

POST alexr-test/_update/1?refresh
{
  "script": {
    "params": {
      "sparse_vector": {
        "c": 3
      }
    },
    "source": "ctx._source.sparse_vector = params.sparse_vector"
  }
}

Logs (if relevant)

No response

[0xShubhamSolanki]

Hi, @spinscale!
Thanks for reporting this. After looking into it, this actually seems like expected behavior rather than a bug. The sparse_vector field is treated like a nested object (a map of token-value pairs), and the Update API's partial update (via the doc parameter) merges objects by default, preserving existing keys and adding new ones. This aligns with how JSON objects are handled in Elasticsearch.
In contrast, the keyword field is a multi-valued array (primitive/leaf type), so it gets fully replaced on update, which explains the difference you observed.
For reference:

Update API docs on partial updates and merging: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html
Sparse vector mapping details: https://www.elastic.co/guide/en/elasticsearch/reference/current/sparse-vector.html

If you need to fully replace the sparse_vector contents, your scripting workaround is spot on—using a script to assign the new value directly works great.
The same likely applies to rank_features since it's also object-like.
If this matches what you were seeing and isn't a bug, feel free to close the issue. If there's more to it or if you'd like a docs improvement to clarify this behavior for sparse vectors, let me know!

[joshua-adams-1]

I'm not sure the best team to route this too, so am forwarding to the Search team as a starting point

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[benwtrent]

Indeed, this is on purpose. If you wish to update particular values, you can do that in the update request. If you want to replace the entire option, scripting is required.

Being able to update individual components within sparse_vector is desirable. Knowing when a user wants to wholly replace the object vs. add/update a particular component is tricky.

I would say this is a feature, rather than a bug. However, we should 100% update our documentation for this field type to make this abundantly clear. I will tag this issue as such.

[elasticsearchmachine]

Pinging @elastic/core-docs (Team:Docs)

[leemthompo]

@benwtrent I happened upon this and had a little window at my EOD so I threw up a PR

#135788