Skip to content

EDOT Collector - exporters/elasticsearch - Duplicate metrics in otel mode #10968

@syepes

Description

@syepes

I'm using the OTel Collector (EDOT) version 9.1.6 to scrape metrics from a Prometheus exporter and forward them to Elastic 9.1.6 in OTel mode. The metrics are being received as expected, but each metric appears twice in the Datastream: once with the prefix "metrics." and once with just the original metric name.

Is there a way to configure the Elastic exporter (elasticsearch) to send only one version of each metric (either with or without the "metrics." prefix), while still using the datastream TSDB mode?
It would be great to store the values one single time to optimize the storage usage.

Image Image

For confirmed bugs, please report:

  • Tested Versions of EDOT: 9.1.6 and 9.2.0
  • Operating System: Elastic Cloud 9.1.6
  • Steps to Reproduce: Below is the used OTel Config
    exporters:
      elasticsearch/otel:
        endpoints: [ "${env:ES_ENDPOINT}" ]
        api_key: ${env:ES_API_KEY}
        mapping:
          mode: otel
        logs_dynamic_index:
          enabled: true
        metrics_dynamic_index:
          enabled: true
        traces_dynamic_index:
          enabled: true
    receivers:
      prometheus/flink/jobmanager:
        config:
          scrape_configs:
            - job_name: 'flink-jobmanager'
              scrape_interval: 30s
              kubernetes_sd_configs:
                - role: pod
              relabel_configs:
                - source_labels: [__meta_kubernetes_pod_label_component]
                  action: keep
                  regex: jobmanager
                - source_labels: [__meta_kubernetes_pod_name]
                  target_label: instance
                - source_labels: [__meta_kubernetes_namespace]
                  target_label: k8s_namespace_name
                - source_labels: [__meta_kubernetes_pod_name]
                  target_label: k8s_pod_name
                - source_labels: [__meta_kubernetes_pod_node_name]
                  target_label: k8s_node_name
                - source_labels: [__address__]
                  action: replace
                  regex: ([^:]+)(?::\d+)?
                  replacement: $1:9249
                  target_label: __address__
                - target_label: __metrics_path__
                  replacement: /metrics
    processors:
      attributes/dataset:
        actions:
          - key: event.dataset
            from_attribute: data_stream.dataset
            action: upsert
      attributes/flink_jobmanager_metrics_dataset:
        actions:
          - key: data_stream.dataset
            value: "flink.jobmanager"
            action: upsert
    service:
      pipelines:
        metrics/flink/jobmanager/otel:
          receivers:
          - prometheus/flink/jobmanager
          processors:
          - attributes/flink_jobmanager_metrics_dataset
          - attributes/dataset
          exporters:
          - elasticsearch/otel

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions