Skip to content

Conversation

masseyke
Copy link
Member

This adds support for a new mapping_addition field to the body of an ingest simulate API request. This mapping addition can be used to change the mappings used for validation. The contents of mapping_addition are merged into the mapping of the index into which the data would be inserted (only for the duration of this request) just before validation.

## First we create a component template with a strict mapping with two fields:
PUT _component_template/mappings_template
{
  "template": {
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "foo": {
          "type": "keyword"
        },
        "bar": {
          "type": "boolean"
        }
      }
    }
  }
}

## Here we create an index template  pulling in the component template above
PUT _index_template/template_1
{
  "index_patterns": ["foo*"],
  "composed_of": ["mappings_template"]
}

## We can index a document here to create the index, or not. Either way the simulate call ought to work the same
POST foo-1/_doc
{
  "foo": "FOO",
  "bar": true
}

## Event though bar is defined as a boolean and there is no baz field in the strict mapping above, this
## will not blow up with validation exceptions because the "mapping_addition" modifies the definition
## of the bar field, and adds a definition of the baz field. The contents of mapping_addition take
## priority over the existing mapping (or any mappings that come from the
component_template_substitutions or index_template_substitutions objects).
POST _ingest/_simulate?pretty&index=foo-1
{
  "docs": [
    {
      "_id": "asdf",
      "_source": {
        "foo": "foo",
        "bar": "bar",
        "baz": "baz"
      }
    }
  ],
  "mapping_addition": {
      "properties": {
        "bar": {
          "type": "keyword"
        },
        "baz": {
          "type": "keyword"
        },
  }
}

@masseyke masseyke added >enhancement :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP backport v8.17.0 auto-merge labels Oct 21, 2024
Copy link
Contributor

Documentation preview:

@masseyke masseyke merged commit 35f7efe into elastic:8.x Oct 22, 2024
16 checks passed
@masseyke masseyke deleted the backport/8.x/pr-114742 branch October 22, 2024 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement v8.17.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant