Create a default mapping template for metrics

[exekias]

With dynamic templates in bulk requests (#69948) merged, we are considering the option of creating a default template to allow for dynamic metrics ingestion. This would be leveraged by many Elastic Agent integrations, and is specially interesting for the ones reporting dynamic metrics (where we only get to know the metric / field names at runtime, and hence, we cannot define a mapping before ingestion).

The overall idea is to map all possible combinations of <metric_type> and <unit>, so each combination gets a unique predictable name that can be referenced at ingest time. For example:

{
  "mappings": {
    "dynamic_templates": [{
      "gauge_double_s": {
        "mapping": {
          "type": "double",
          "meta": {
            "metric_type": "gauge",
            "unit": "s"
          }
        }
      }
    },
    {
      "gauge_double_byte": {
        "mapping": {
          "type": "double",
          "meta": {
            "metric_type": "gauge",
            "unit": "byte"
          }
        }
      }
    },
    ...]
  }
}

The obvious concern is: We would be creating a dynamic mapping entry per every combination of <metric_type> and , which will explode into many entries:

• metric_type: gauge, counter, histogram. Not expected to grow much
• unit: byte, percent, d, h, m, s, ms, micros, nanos, with more units to come as we onboard other metrics.
• ES field type: long, double, integer, byte, float, scaled_float.

So the main question is: Would this be considered a good practice? Can it cause any issues because of a too large mapping template? Perhaps we should do this in a different way?

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[elasticmachine]

Pinging @elastic/es-core-features (Team:Core/Features)

[jpountz]

@exekias I can imagine the number of dynamic templates growing very quickly using this approach so I'd like to take some time to look into whether Elasticsearch can make it easier.

One idea I'm considering is giving clients the ability to override metadata as part of bulk requests similarly to how we allow selecting a dynamic template, something like this:

POST /_bulk
{ "create" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"system.cpu.user.pct": "scaled_float_100"}, "meta": {"system.cpu.user.pct": { "unit": "percent", "metric_type": "gauge" }}} }
{ "system": { "cpu": { "user": { "pct": "22.7" }}}}

With the dynamic_templates option, we avoided giving the ability to pass arbitrary mappings as otherwise users who only have the create_doc and auto_configure privileges could effectively put arbitrary mappings even though they don't have the manage privilege. Maybe it's ok to make an exception for the meta of mappings given that it never affects how fields get indexed. But then we'd need to make sure it remains that way.

cc @csoulios given the ongoing discussion about leveraging meta to configure rollups
cc @tvernum do you have an opinion about the security implications?

[tvernum]

Because other stack components rely on meta being meaningful, I think there are some implications if we allow all users to set the metadata too freely.
Or more accurately, I'm not confident that we can be sure there are no implications. It might be fine, but the system is sufficiently complex that it's hard to be sure.

I'd feel better if the definition of those dynamic templates had some way to indicate that they expected parameterized metadata, either a new flag (accept_custom_meta), or by making use of template variables in some way.

[axw]

or by making use of template variables in some way.

I really like this idea. I imagine a request would look something like:

POST /_bulk
{ "create" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"system.cpu.user.pct": {"name": "scaled_float", "params": {"scaling_factor": 100, "metric_type": "gauge", "unit": "percent"}}} } }
{ "system": { "cpu": { "user": { "pct": "22.7" }}}}

and the dynamic_template (but probably with some conditional rendering of meta):

"scaled_float": {
  "mapping": {
    "type": "scaled_float",
    "scaling_factor": {{scaling_factor}},
    "meta": {
      "metric_type": {{metric_type}},
      "unit": {{unit}}
    }
  }
}