Configuring EMF exporter Dimensions

[Konig-Corey]

There are a few options out of the box that the emf exporter offers for metric rollup. Aside from that there is an option to manually set the dimension sets you would like to use when sending to clouddwatch. This custom dimension configuration works great for us except I would like to be able to make these custom dimensions sets be used in addition to the default dimension set which is all fields.

For example, If you have the following:

exporters:
  awsemf:
    namespace: "my-testing-namespace"
    dimension_rollup_option: "NoDimensionRollup"
    resource_to_telemetry_conversion:
      enabled: true

This will publish cloudwatch metrics with one "dimension" which includes all of the fields sent like shown below

{
    "_aws": {
        "CloudWatchMetrics": [
            {
                "Namespace": "exp-h-coreysvcs",
                "Dimensions": [
                    [
                        "cpu",
                        "state",
                        "asg",
                        "pod",
                        "host-id",
                        "service"
                    ]
                ],
                "Metrics": [
                    {
                        "Name": "system.cpu.time",
                        "Unit": "Seconds"
                    }
                ]
            }
        ],
        "Timestamp": 1629317716190
    },
    "asg": "my-asg",
    "cpu": "cpu1",
    "host-id": "i-coreytestinghostid",
    "pod": "a",
    "service": "coreysvcs",
    "state": "nice",
    "system.cpu.time": 0
}

vs the following configuration:

exporters:
  awsemf:
    namespace: "exp-h-coreysvcs"
    dimension_rollup_option: "NoDimensionRollup"
    resource_to_telemetry_conversion:
      enabled: true
    metric_declarations:
      dimensions:
        - ["asg"]
        - ["asg", "host-id"]
        - []
      metric_name_selectors:
        - ".*"

which will yield the following:

{
    "_aws": {
        "CloudWatchMetrics": [
            {
                "Namespace": "exp-h-coreysvcs",
                "Dimensions": [
                    [
                        "asg"
                    ],
                    [
                        "asg",
                        "host-id"
                    ]
                ],
                "Metrics": [
                    {
                        "Name": "system.cpu.time",
                        "Unit": "Seconds"
                    }
                ]
            }
        ],
        "Timestamp": 1629320444164
    },
    "asg": "my-asg",
    "cpu": "cpu0",
    "host-id": "i-coreytestinghostid",
    "pod": "a",
    "service": "coreysvcs",
    "state": "nice",
    "system.cpu.time": 0.6499999999996362
}

How can I send a dimension of everything AND my custom dimensions?

For additional context. The reason I am choosing to only send the dimensions I want vs using the roll up features is the roll up features generate a bunch of variants of the metrics that are not needed. I want to have one dimension of all fields + a set of custom dimensions of my choosing so that I can control the exact dimensions that are created in cloudwatch vs an all or nothing tactic with the roll up options.

[mayur-oci]

@Konig-Corey I can not help with your query definitely but may be you can help me with my question.

I have a conceptual question related to AWS EMF Exporter for OTEL(https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/awsemfexporter).,
AFAIK AWS CW metrics are just time series of data points, then how does OTEL metrics of type say Histogram are modeled in AWS CW?
is there any conversion done by AWS EMF Exporter here, to make OTEL Metrics compatible with AWS CW?

Thanks in Advance

[jpmatusik]

Hi. Super late but this post came up in my google search now so 🤷 . If i'm reading the question correctly, this is possible if you configure two exporters. Hopefully that helps.

exporters:
  awsemf/all:
    namespace: "exp-h-coreysvcs"
    dimension_rollup_option: "NoDimensionRollup"
    resource_to_telemetry_conversion:
      enabled: true
  awsemf/custom:
    namespace: "exp-h-coreysvcs"
    dimension_rollup_option: "NoDimensionRollup"
    resource_to_telemetry_conversion:
      enabled: true
    metric_declarations:
      dimensions:
        - ["asg"]
        - ["asg", "host-id"]
      metric_name_selectors:
        - ".*"
...
service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [awsemf/all, awsemf/custom]