Add OTLP metrics endpoint

[felixbarny]

Adds an OTLP endpoint for metric ingestion into TSDB.

This simplifies getting data into TSDB as configuration of mappings and dimensions isn't required. It also improves ingestion performance. Partly due to the binary encoding but also because it can re-use partial TsidBuilders for common resource attributes.

My aim was to retain exactly the same semantics and behavior as the OTel collector's Elasticsearch exporter, including data stream routing and mapping hints.

Depends on

• Optimize IP field parsing #132463
• TSDB ingest performance: combine routing and tsdb hashing #132566

To review, select from the 3rd commit (a46882e) onwards
Note that I might force-push to the branch once the PRs this depends on get merged.

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[ruflin]

Did a quick test, and so far seems to be working as expected. For everyone that needs a config to do a quick test run with the otel collector:

receivers:
  hostmetrics:
    collection_interval: 10s
    scrapers:
      cpu:
      disk:
      load:
      filesystem:
      memory:
      network:
      #process:
      paging:

processors:
  batch:


exporters:
  otlphttp/elasticsearch:
    compression: gzip
    encoding: proto
    endpoint: http://localhost:9200/_otlp
    sending_queue:
      enabled: true
      block_on_overflow: true
      queue_size: 10
      num_consumers: 10

service:
  pipelines:
    metrics:
      receivers: [hostmetrics]
      processors: [batch]
      exporters: [otlphttp/elasticsearch] # only use with batch processor