Bug: Custom Metrics with different Dimensions treated as Multi-value Metrics when using `log_metrics` Decorator.

[rastogin-rp]

Expected Behaviour

While using the AWS Powertools for Metrics, I've encountered an issue with how multiple metrics EMFs are handled when they share the same name but differ by their dimensions while using the log_metrics decorator.

Current Behaviour

We have a custom high-resolution Lambda Execute metric where a custom dimension called Type alternates between the values Start and End.

The package currently treats these as multi-value metrics, combining them into one metric EMF when log metrics are called, even though they represent distinct operations and should be treated as separate EMFs with individual timestamps.

Code snippet

Given the following setup:

from aws_lambda_powertools import Metrics

metrics = Metrics(namespace="MyApp", service="SomeLambda")

@metrics.log_metrics
def lambda_handler(event, context):
    # Metric for starting the lambda execution
    metrics.add_metric(name="Lambda Execute", unit="None", value=1, resolution=1)
    metrics.add_dimension(name="Type", value="Start")

    # Business logic or operations
    # ...

    # Metric for completing the lambda execution
    metrics.add_metric(name="Lambda Execute", unit="None", value=1, resolution=1)
    metrics.add_dimension(name="Type", value="End")
    
    return {
        'statusCode': 200,
        'body': "Execution complete"
    }

I should have received two EMF logs:

{"_aws":{"Timestamp":1728566819588,"CloudWatchMetrics":[{"Namespace":"MyApp","Dimensions":[["Type"]],"Metrics":[{"Name":"Lambda Execute","Unit":"None","StorageResolution":1}]}]},"Type":"Start","Lambda Execute":[1.0]}

...some other logs from code...

{"_aws":{"Timestamp":1728566822038,"CloudWatchMetrics":[{"Namespace":"MyApp","Dimensions":[["Type"]],"Metrics":[{"Name":"Lambda Execute","Unit":"None","StorageResolution":1}]}]},"Type":"End","Lambda Execute":[1.0]}

But instead, I get only one EMF as a multi-value metric as follows:

{"_aws":{"Timestamp":1728566822038,"CloudWatchMetrics":[{"Namespace":"MyApp","Dimensions":[["Type"]],"Metrics":[{"Name":"Lambda Execute","Unit":"None","StorageResolution":1}]}]},"Type":"End","Lambda Execute":[1.0, 1.0]}

Possible Solution

Add logic to check for varying dimension values to identify whether it's a unique metric or multi-value metric EMF.

Steps to Reproduce

1. Define two Lambda Execute custom metrics with a Type dimension alternating between start and end.
2. Use the AWS Powertools metrics decorator on the Lambda function to flush metrics.
3. Notice that it generates a single EMF with multiple values, with the last set type dimension value, instead of two distinct EMFs as one would expect in the logs.

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.12

Packaging format used

PyPi

Debugging logs

No response

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[leandrodamascena]

Hey @rastogin-rp! Thanks for opening this issue! You're reporting a curious behavior and I'll need to read to confirm to confirm somethings. I'm doing this right now and as soon as I have any news, I will come back here with some update.

[leandrodamascena]

Hello @rastogin-rp! I was checking the code and the reported behavior and I have some considerations. I don't consider this a bug, this is the expected behavior by design. Let me detail some points and explain better.

1 - You are using the same Metrics instance to publish a metric called Lambda Execute in two different parts of your script, which is completely fine and we are serializing the metric correctly, that is, creating 2 values ​​for this metric, but with the same timestamp, which you may not want.

2 - Explaining the mechanism of how we flush the metrics: We store the metrics in a buffer and when you use the @metrics.log_metrics decorator we automatically flush the metrics in 2 circumstances 1/ when the function is executed 2/ when the buffer reaches 100 entries (EMF limit), whichever comes first.

3 - The other part is that you are trying to add two values ​​to the same dimension called Type and this will not work as expected with the current code. Dimensions is a array of string and you cannot have multiple values ​​for the same Dimension, in your case the last value for Type will prevail, which in this case is End. Remember topic 2 where I explain that we buffer the metrics. Maybe on our part it is necessary to put a warning for situations like this, which would already help in the feedback loop and catch the error early.

4 - To make this work in your case you have 2 options 1/ use Ephemeral Metrics and then you can have complete isolation between metrics creation, or you can flush the metrics manually, what I think is more indicated here and you and use the same code you have and make small changes. Let me send an example:

from aws_lambda_powertools import Metrics

metrics = Metrics(namespace="MyApp", service="SomeLambda")

@metrics.log_metrics
def lambda_handler(event, context):
    # Metric for starting the lambda execution
    metrics.add_dimension(name="Type", value="Start")
    metrics.add_metric(name="Lambda Execute", unit="None", value=1, resolution=1)
    metrics.flush_metrics() # YOU'RE FLUSHING manually the first metrics and cleaning the buffer, including Dimension.
    

    # Business logic or operations
    # ...

    # Metric for completing the lambda execution
    #

    # NOW METRICS IS CLEAN and you're create a new metric with the same name, but different timestamp and dimensions.
    metrics.add_metric(name="Lambda Execute", unit="None", value=1, resolution=1)
    metrics.add_dimension(name="Type", value="End")
    # DON'T NEED TO FLUSH METRIC HERE BECAUSE THE DECORATOR WILL

    #raise 
    return {
        'statusCode': 200,
        'body': "Execution complete"
    }

We buffer metrics for optimization, but in this case I don't see a problem in flushing the first metric manually. I don't think there will be any latency here.

Please let me know if this helps.

Leandro

[leandrodamascena]

I'm closing this issue. Please reopen if you need any additional help @rastogin-rp!