Skip to content

feat: reimplement history data export #1642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Sep 22, 2025
Merged

feat: reimplement history data export #1642

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
6401f0f
feat: reimplement history data export
woehrl01 Feb 25, 2025
e089bca
fix: use struct directly instead of referenes to reduce gc preassure
woehrl01 Feb 27, 2025
ce858fb
improve comments and add example
woehrl01 Feb 28, 2025
1d3536c
rename to exportAllDataPoints and added test
woehrl01 Mar 7, 2025
96e5285
correctly handle nil value and timestamp combinations
woehrl01 Mar 7, 2025
52f7b4e
Code review feedback
kgeckhart Jul 18, 2025
88404b3
Merge pull request #1 from kgeckhart/historical-data
woehrl01 Jul 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 26 additions & 2 deletions docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,14 @@ statistics:
# Export the metric with the original CloudWatch timestamp (General Setting for all metrics in this job)
[ addCloudwatchTimestamp: <boolean> ]

# Enables the inclusion of past metric data points from the CloudWatch response if available.
# This is useful when a metric is configured with a 60-second period and a 300-second duration, ensuring that all
# five data points are exposed at the metrics endpoint instead of only the latest one.
# Note: This option requires `addCloudwatchTimestamp` to be enabled.
# The metric destination must support out of order timestamps, see https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tsdb
# (General Setting for all metrics in this job)
[ exportAllDataPoints: <boolean> ]

# List of metric definitions
metrics:
[ - <metric_config> ... ]
Expand Down Expand Up @@ -276,6 +284,14 @@ statistics:
# Export the metric with the original CloudWatch timestamp (General Setting for all metrics in this job)
[ addCloudwatchTimestamp: <boolean> ]

# Enables the inclusion of past metric data points from the CloudWatch response if available.
# This is useful when a metric is configured with a 60-second period and a 300-second duration, ensuring that all
# five data points are exposed at the metrics endpoint instead of only the latest one.
# Note: This option requires `addCloudwatchTimestamp` to be enabled.
# The metric destination must support out of order timestamps, see https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tsdb
# (General Setting for all metrics in this job)
[ exportAllDataPoints: <boolean> ]

# List of metric definitions
metrics:
[ - <metric_config> ... ]
Expand Down Expand Up @@ -333,12 +349,20 @@ statistics:

# Export the metric with the original CloudWatch timestamp (Overrides job level setting)
[ addCloudwatchTimestamp: <boolean> ]

# Enables the inclusion of past metric data points from the CloudWatch response if available.
# This is useful when a metric is configured with a 60-second period and a 300-second duration, ensuring that all
# five data points are exposed at the metrics endpoint instead of only the latest one.
# Note: This option requires `addCloudwatchTimestamp` to be enabled.
# The metric destination must support out of order timestamps, see https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tsdb
# (General Setting for all metrics in this job)
[ exportAllDataPoints: <boolean> ]
```

Notes:
- Available statistics: `Maximum`, `Minimum`, `Sum`, `SampleCount`, `Average`, `pXX` (e.g. `p90`).

- Watch out using `addCloudwatchTimestamp` for sparse metrics, e.g from S3, since Prometheus won't scrape metrics containing timestamps older than 2-3 hours.
- Watch out using `addCloudwatchTimestamp` for sparse metrics, e.g from S3, since Prometheus won't scrape metrics containing timestamps older than 2-3 hours. Also the same applies when enabling `exportAllDataPoints` in any metric.

### `exported_tags_config`

Expand Down Expand Up @@ -390,4 +414,4 @@ This is an example of the `dimensions_config` block:
dimensions:
- name: AutoScalingGroupName
value: MyGroup
```
```
29 changes: 29 additions & 0 deletions examples/historic-data.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
apiVersion: v1alpha1
discovery:
jobs:
- type: AWS/SQS
regions:
- us-east-1
period: 60
length: 300
addCloudwatchTimestamp: true
exportAllDataPoints: true
metrics:
- name: NumberOfMessagesSent
statistics: [Sum]
- name: NumberOfMessagesReceived
statistics: [Sum]
- name: NumberOfMessagesDeleted
statistics: [Sum]
- name: ApproximateAgeOfOldestMessage
statistics: [Average]
- name: NumberOfEmptyReceives
statistics: [Sum]
- name: SentMessageSize
statistics: [Average]
- name: ApproximateNumberOfMessagesNotVisible
statistics: [Sum]
- name: ApproximateNumberOfMessagesDelayed
statistics: [Sum]
- name: ApproximateNumberOfMessagesVisible
statistics: [Sum]
13 changes: 8 additions & 5 deletions pkg/clients/cloudwatch/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ type Client interface {
GetMetricData(ctx context.Context, getMetricData []*model.CloudwatchData, namespace string, startTime time.Time, endTime time.Time) []MetricDataResult

// GetMetricStatistics returns the output of the GetMetricStatistics CloudWatch API.
GetMetricStatistics(ctx context.Context, logger *slog.Logger, dimensions []model.Dimension, namespace string, metric *model.MetricConfig) []*model.Datapoint
GetMetricStatistics(ctx context.Context, logger *slog.Logger, dimensions []model.Dimension, namespace string, metric *model.MetricConfig) []*model.MetricStatisticsResult
}

// ConcurrencyLimiter limits the concurrency when calling AWS CloudWatch APIs. The functions implemented
Expand All @@ -55,9 +55,12 @@ type ConcurrencyLimiter interface {
}

type MetricDataResult struct {
ID string
// A nil datapoint is a marker for no datapoint being found
Datapoint *float64
ID string
DataPoints []DataPoint
}

type DataPoint struct {
Value *float64
Timestamp time.Time
}

Expand All @@ -73,7 +76,7 @@ func NewLimitedConcurrencyClient(client Client, limiter ConcurrencyLimiter) Clie
}
}

func (c limitedConcurrencyClient) GetMetricStatistics(ctx context.Context, logger *slog.Logger, dimensions []model.Dimension, namespace string, metric *model.MetricConfig) []*model.Datapoint {
func (c limitedConcurrencyClient) GetMetricStatistics(ctx context.Context, logger *slog.Logger, dimensions []model.Dimension, namespace string, metric *model.MetricConfig) []*model.MetricStatisticsResult {
c.limiter.Acquire(getMetricStatisticsCall)
res := c.client.GetMetricStatistics(ctx, logger, dimensions, namespace, metric)
c.limiter.Release(getMetricStatisticsCall)
Expand Down
34 changes: 22 additions & 12 deletions pkg/clients/cloudwatch/v1/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ func toModelDimensions(dimensions []*cloudwatch.Dimension) []model.Dimension {

func (c client) GetMetricData(ctx context.Context, getMetricData []*model.CloudwatchData, namespace string, startTime time.Time, endTime time.Time) []cloudwatch_client.MetricDataResult {
metricDataQueries := make([]*cloudwatch.MetricDataQuery, 0, len(getMetricData))
exportAllDataPoints := false
for _, data := range getMetricData {
metricStat := &cloudwatch.MetricStat{
Metric: &cloudwatch.Metric{
Expand All @@ -110,6 +111,7 @@ func (c client) GetMetricData(ctx context.Context, getMetricData []*model.Cloudw
MetricStat: metricStat,
ReturnData: aws.Bool(true),
})
exportAllDataPoints = exportAllDataPoints || data.MetricMigrationParams.ExportAllDataPoints
}
input := &cloudwatch.GetMetricDataInput{
EndTime: &endTime,
Expand Down Expand Up @@ -137,23 +139,31 @@ func (c client) GetMetricData(ctx context.Context, getMetricData []*model.Cloudw
c.logger.Error("GetMetricData error", "err", err)
return nil
}
return toMetricDataResult(resp)
return toMetricDataResult(resp, exportAllDataPoints)
}

func toMetricDataResult(resp cloudwatch.GetMetricDataOutput) []cloudwatch_client.MetricDataResult {
func toMetricDataResult(resp cloudwatch.GetMetricDataOutput, exportAllDataPoints bool) []cloudwatch_client.MetricDataResult {
output := make([]cloudwatch_client.MetricDataResult, 0, len(resp.MetricDataResults))
for _, metricDataResult := range resp.MetricDataResults {
mappedResult := cloudwatch_client.MetricDataResult{ID: *metricDataResult.Id}
if len(metricDataResult.Values) > 0 {
mappedResult.Datapoint = metricDataResult.Values[0]
mappedResult.Timestamp = *metricDataResult.Timestamps[0]
mappedResult := cloudwatch_client.MetricDataResult{
ID: *metricDataResult.Id,
DataPoints: make([]cloudwatch_client.DataPoint, 0, len(metricDataResult.Timestamps))}
for i := 0; i < len(metricDataResult.Timestamps); i++ {
mappedResult.DataPoints = append(mappedResult.DataPoints, cloudwatch_client.DataPoint{
Value: metricDataResult.Values[i],
Timestamp: *metricDataResult.Timestamps[i],
})

if !exportAllDataPoints {
break
}
}
output = append(output, mappedResult)
}
return output
}

func (c client) GetMetricStatistics(ctx context.Context, logger *slog.Logger, dimensions []model.Dimension, namespace string, metric *model.MetricConfig) []*model.Datapoint {
func (c client) GetMetricStatistics(ctx context.Context, logger *slog.Logger, dimensions []model.Dimension, namespace string, metric *model.MetricConfig) []*model.MetricStatisticsResult {
filter := createGetMetricStatisticsInput(dimensions, &namespace, metric, logger)

c.logger.Debug("GetMetricStatistics", "input", filter)
Expand All @@ -171,14 +181,14 @@ func (c client) GetMetricStatistics(ctx context.Context, logger *slog.Logger, di
return nil
}

return toModelDatapoints(resp.Datapoints)
return toModelDataPoints(resp.Datapoints)
}

func toModelDatapoints(cwDatapoints []*cloudwatch.Datapoint) []*model.Datapoint {
modelDataPoints := make([]*model.Datapoint, 0, len(cwDatapoints))
func toModelDataPoints(cwDataPoints []*cloudwatch.Datapoint) []*model.MetricStatisticsResult {
modelDataPoints := make([]*model.MetricStatisticsResult, 0, len(cwDataPoints))

for _, cwDatapoint := range cwDatapoints {
modelDataPoints = append(modelDataPoints, &model.Datapoint{
for _, cwDatapoint := range cwDataPoints {
modelDataPoints = append(modelDataPoints, &model.MetricStatisticsResult{
Average: cwDatapoint.Average,
ExtendedStatistics: cwDatapoint.ExtendedStatistics,
Maximum: cwDatapoint.Maximum,
Expand Down
64 changes: 57 additions & 7 deletions pkg/clients/cloudwatch/v1/client_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,13 @@ func Test_toMetricDataResult(t *testing.T) {
name string
getMetricDataOutput cloudwatch.GetMetricDataOutput
expectedMetricDataResults []cloudwatch_client.MetricDataResult
exportAllDataPoints bool
}

testCases := []testCase{
{
name: "all metrics present",
name: "all metrics present",
exportAllDataPoints: false,
getMetricDataOutput: cloudwatch.GetMetricDataOutput{
MetricDataResults: []*cloudwatch.MetricDataResult{
{
Expand All @@ -67,12 +69,21 @@ func Test_toMetricDataResult(t *testing.T) {
},
},
expectedMetricDataResults: []cloudwatch_client.MetricDataResult{
{ID: "metric-1", Datapoint: aws.Float64(1.0), Timestamp: ts.Add(10 * time.Minute)},
{ID: "metric-2", Datapoint: aws.Float64(2.0), Timestamp: ts},
{
ID: "metric-1", DataPoints: []cloudwatch_client.DataPoint{
{Value: aws.Float64(1.0), Timestamp: ts.Add(10 * time.Minute)},
},
},
{
ID: "metric-2", DataPoints: []cloudwatch_client.DataPoint{
{Value: aws.Float64(2.0), Timestamp: ts},
},
},
},
},
{
name: "metric with no values",
name: "metric with no values",
exportAllDataPoints: false,
getMetricDataOutput: cloudwatch.GetMetricDataOutput{
MetricDataResults: []*cloudwatch.MetricDataResult{
{
Expand All @@ -88,15 +99,54 @@ func Test_toMetricDataResult(t *testing.T) {
},
},
expectedMetricDataResults: []cloudwatch_client.MetricDataResult{
{ID: "metric-1", Datapoint: aws.Float64(1.0), Timestamp: ts.Add(10 * time.Minute)},
{ID: "metric-2", Datapoint: nil, Timestamp: time.Time{}},
{
ID: "metric-1", DataPoints: []cloudwatch_client.DataPoint{
{Value: aws.Float64(1.0), Timestamp: ts.Add(10 * time.Minute)},
},
},
{
ID: "metric-2",
DataPoints: []cloudwatch_client.DataPoint{},
},
},
},
{
name: "export all data points",
exportAllDataPoints: true,
getMetricDataOutput: cloudwatch.GetMetricDataOutput{
MetricDataResults: []*cloudwatch.MetricDataResult{
{
Id: aws.String("metric-1"),
Values: []*float64{aws.Float64(1.0), aws.Float64(2.0), aws.Float64(3.0)},
Timestamps: []*time.Time{aws.Time(ts.Add(10 * time.Minute)), aws.Time(ts.Add(5 * time.Minute)), aws.Time(ts)},
},
{
Id: aws.String("metric-2"),
Values: []*float64{aws.Float64(2.0)},
Timestamps: []*time.Time{aws.Time(ts)},
},
},
},
expectedMetricDataResults: []cloudwatch_client.MetricDataResult{
{
ID: "metric-1", DataPoints: []cloudwatch_client.DataPoint{
{Value: aws.Float64(1.0), Timestamp: ts.Add(10 * time.Minute)},
{Value: aws.Float64(2.0), Timestamp: ts.Add(5 * time.Minute)},
{Value: aws.Float64(3.0), Timestamp: ts},
},
},
{
ID: "metric-2", DataPoints: []cloudwatch_client.DataPoint{
{Value: aws.Float64(2.0), Timestamp: ts},
},
},
},
},
}

for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
metricDataResults := toMetricDataResult(tc.getMetricDataOutput)
metricDataResults := toMetricDataResult(tc.getMetricDataOutput, tc.exportAllDataPoints)
require.Equal(t, tc.expectedMetricDataResults, metricDataResults)
})
}
Expand Down
35 changes: 23 additions & 12 deletions pkg/clients/cloudwatch/v2/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ func toModelDimensions(dimensions []types.Dimension) []model.Dimension {

func (c client) GetMetricData(ctx context.Context, getMetricData []*model.CloudwatchData, namespace string, startTime time.Time, endTime time.Time) []cloudwatch_client.MetricDataResult {
metricDataQueries := make([]types.MetricDataQuery, 0, len(getMetricData))
exportAllDataPoints := false
for _, data := range getMetricData {
metricStat := &types.MetricStat{
Metric: &types.Metric{
Expand All @@ -113,6 +114,7 @@ func (c client) GetMetricData(ctx context.Context, getMetricData []*model.Cloudw
MetricStat: metricStat,
ReturnData: aws.Bool(true),
})
exportAllDataPoints = exportAllDataPoints || data.MetricMigrationParams.ExportAllDataPoints
}

input := &cloudwatch.GetMetricDataInput{
Expand Down Expand Up @@ -143,23 +145,32 @@ func (c client) GetMetricData(ctx context.Context, getMetricData []*model.Cloudw

c.logger.Debug("GetMetricData", "output", resp)

return toMetricDataResult(resp)
return toMetricDataResult(resp, exportAllDataPoints)
}

func toMetricDataResult(resp cloudwatch.GetMetricDataOutput) []cloudwatch_client.MetricDataResult {
func toMetricDataResult(resp cloudwatch.GetMetricDataOutput, exportAllDataPoints bool) []cloudwatch_client.MetricDataResult {
output := make([]cloudwatch_client.MetricDataResult, 0, len(resp.MetricDataResults))
for _, metricDataResult := range resp.MetricDataResults {
mappedResult := cloudwatch_client.MetricDataResult{ID: *metricDataResult.Id}
if len(metricDataResult.Values) > 0 {
mappedResult.Datapoint = &metricDataResult.Values[0]
mappedResult.Timestamp = metricDataResult.Timestamps[0]
mappedResult := cloudwatch_client.MetricDataResult{
ID: *metricDataResult.Id,
DataPoints: make([]cloudwatch_client.DataPoint, 0, len(metricDataResult.Timestamps)),
}
for i := 0; i < len(metricDataResult.Timestamps); i++ {
Copy link
Contributor

@cristiangreco cristiangreco Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this new behaviour should be used only if exportAllDataPoints is enabled. If not, we should keep the previous behaviour and avoid copying around the additional values/timestamps from the CW response.

Copy link
Contributor Author

@woehrl01 woehrl01 Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required because this function is not aware of any configuration. This is in my opinion the cleaned implementation. See comments from @kgeckhart in the linked pr

Copy link
Contributor

@cristiangreco cristiangreco Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Kyle's comment was mainly around allowing the client to return a slice of Datapoint + Timestamp, but he can confirm.

My concern is around useless mem allocations when exportAllDataPoints is not enabled. I think the option can be made available to the aws client, e.g. via CloudwatchData or GetMetricDataProcessingParams.

Copy link
Contributor Author

@woehrl01 woehrl01 Mar 29, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure there is no significant useless memory allocation. All the values are already in memory because it's part of the api response. So we just allocate the slice a bit bigger, because we use a struct of slice and not a reference to the struct slice, there is a single memory alloc for the underlying array, everything else is just copying over values. So this just keeps the already returned data for a few function calls longer in memory, to unify the struct across v1 and v2.

Feels like an unessary add of complexity, to pass around those parameters and adding additional tests for validate that behaviour.

Copy link
Contributor Author

@woehrl01 woehrl01 Mar 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on my earlier comment: even if we assume returning a slice of 5 structs increases memory usage, the overhead is still reasonable, around ~128 MB for 1 million metrics. That comes from storing 4 additional data points per metric, each at 32 bytes (a time.Time value + a *float64 pointer). It's just a larger backing array and a few more pointers to scan during GC. This only becomes relevant if we're allocating and retaining millions of these slices, which we're not.

And if we were dealing with that many, querying 1 million CloudWatch metrics, the real issue wouldn't be memory. Even with 5 data points per request, that's 200K metric fetches, costing ~$2 per query. At that scale, CloudWatch costs and API limits are a far bigger concern than a ~100 MB memory difference.

Copy link
Contributor

@kgeckhart kgeckhart Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We run YACE in a stateless multi-tenant fashion so querying a very large amount of metrics in a short period of time often happens. I was concerned about the potential memory overhead for this change and had shared that with Cristian.

This wasn't the area I was concerned about initially as we keep our length + period setup to only produce a single datapoint as much as possible. I do agree with Cristian, if we stick to only mapping a single datapoint when the setting is disabled it will ensure there's a minimal overhead to those who upgrade to latest without using the feature.

I was primarily concerned about overhead from switching to a single datapoint -> a slice for our use case. I don't think we should do anything about it now. I wrote a OneOrMany[T] that benchmarks nicely vs a single entry slice and would PR it separately if needed.

Copy link
Contributor Author

@woehrl01 woehrl01 Jul 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kgeckhart I just blindly merged yesterday your suggested changes, to get this PR of my plate. Still I had a look today, about what you changed around your memory concerns. Looking at the current changes, I don't see any change which would reduce memory usage at all. Despite your suggestion, it still creates a slice everytime and it always creates it with full size for all data points. There is now just added complexity around passing over a flag, to stop the loop early, which you could argue reduced CPU overhead, but is likely neglectable at the scale where the cloudwatch costs would explode.

Before we are moving into premature optimisations, have you actually measured and proved that this actually is an issue? (see my comments above why I doubt that based on numbers). You mention that you are running this at large scale. Maybe you can run the initial PR, side by side for 1-5% of your workload (depending on the scale) and share some realworld memory impact?

Copy link
Contributor

@kgeckhart kgeckhart Jul 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@woehrl01 thanks for taking a look at it and providing feedback.

Despite your suggestion, it still creates a slice everytime and it always creates it with full size for all data points.

This was an oversight on my part as I was hastily making the change before I needed to go to an event yesterday 😅. My intention was to make sure to only allocate exactly what is necessary and nothing more which I can do with another minor change

There is now just added complexity around passing over a flag, to stop the loop early, which you could argue reduced CPU overhead, but is likely neglectable at the scale where the cloudwatch costs would explode.

IMO the added complexity is rather minimal and easily testable which is why I went through with the change.

Before we are moving into premature optimisations, have you actually measured and proved that this actually is an issue? (see my comments above why I doubt that based on numbers). You mention that you are running this at large scale. Maybe you can run the initial PR, side by side for 1-5% of your workload (depending on the scale) and share some realworld memory impact?

When Cristian brought it up I didn't quite get it at first because we do run it at scale but we do it in such a way that we should only ever get one data point back. We do this intentionally for performance reasons, why ask for data you won't use? I don't know if it was Cristian's intent but my realization was that introducing this feature should not present a noticeable negative impact on the larger community just by upgrading. This exporter is embedded in to places like https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.exporter.cloudwatch/ which I know is used by customers at a scale large enough to incur some rather hefty CloudWatch costs.

Going from a single data point to a slice presents some memory increase that is unlikely to be noticeable for most. We don't know how many people are setting up their configs with a length that is larger than their period (length > period = more than 1 data point) and to what degree they are doing it (2x, 3x, 10x?). This is part of the complexity of building the configs CloudWatch has some incredibly odd behaviors for different metrics so the configs get cargo culted in a way that is not optimal but works.

Could this guard be unnecessary, yes but the complexity of adding it feels acceptable as a means to try to provide a stable experience to existing users.

Since I messed up the DCO + need another change we will have to figure out what to do with this PR. But first it would probably be good to have @cristiangreco take a look to make sure there's nothing further.

Copy link

@ecktom ecktom Aug 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kgeckhart @cristiangreco what's the plan with this PR?
Don't get me wrong, I understand we're all busy and this is OSS after all. So no pressure here please... ;) I would just highly appreciate some short indication of you guys if and when you plan to move ahead with this PR to be able to make plans on our end as well.

From what I see, this PR seems to be waiting on some action from your side and there is nothing really left the community could support with? If that is wrong and there is still something todo, please let me know, I'm happy to contribute as well.

mappedResult.DataPoints = append(mappedResult.DataPoints, cloudwatch_client.DataPoint{
Value: &metricDataResult.Values[i],
Timestamp: metricDataResult.Timestamps[i],
})

if !exportAllDataPoints {
break
}
}
output = append(output, mappedResult)
}
return output
}

func (c client) GetMetricStatistics(ctx context.Context, logger *slog.Logger, dimensions []model.Dimension, namespace string, metric *model.MetricConfig) []*model.Datapoint {
func (c client) GetMetricStatistics(ctx context.Context, logger *slog.Logger, dimensions []model.Dimension, namespace string, metric *model.MetricConfig) []*model.MetricStatisticsResult {
filter := createGetMetricStatisticsInput(logger, dimensions, &namespace, metric)
c.logger.Debug("GetMetricStatistics", "input", filter)

Expand All @@ -181,18 +192,18 @@ func (c client) GetMetricStatistics(ctx context.Context, logger *slog.Logger, di
ptrs = append(ptrs, &datapoint)
}

return toModelDatapoints(ptrs)
return toModelDataPoints(ptrs)
}

func toModelDatapoints(cwDatapoints []*types.Datapoint) []*model.Datapoint {
modelDataPoints := make([]*model.Datapoint, 0, len(cwDatapoints))
func toModelDataPoints(cwDataPoints []*types.Datapoint) []*model.MetricStatisticsResult {
modelDataPoints := make([]*model.MetricStatisticsResult, 0, len(cwDataPoints))

for _, cwDatapoint := range cwDatapoints {
for _, cwDatapoint := range cwDataPoints {
extendedStats := make(map[string]*float64, len(cwDatapoint.ExtendedStatistics))
for name, value := range cwDatapoint.ExtendedStatistics {
extendedStats[name] = &value
}
modelDataPoints = append(modelDataPoints, &model.Datapoint{
modelDataPoints = append(modelDataPoints, &model.MetricStatisticsResult{
Average: cwDatapoint.Average,
ExtendedStatistics: extendedStats,
Maximum: cwDatapoint.Maximum,
Expand Down
Loading