Skip to content

feat(prometheus.exporter.azure): expose concurrency settings #4951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
akselleirv wants to merge 2 commits into
base: main
Choose a base branch
from
+164 −75
Open

feat(prometheus.exporter.azure): expose concurrency settings #4951

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bb53c49
feat(prometheus.exporter.azure): expose concurrency settings
akselleirv Nov 28, 2025
3229e59
Update docs/sources/reference/components/prometheus/prometheus.export…
akselleirv Nov 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Main (unreleased)

- Added `send_traceparent` option for `tracing` config to enable traceparent header propagation. (@MyDigitalLife)

- `prometheus.exporter.azure` now supports `concurrency_subscription` and `concurrency_subscription_resource` arguments to control the number of concurrent API requests. (@akselleirv)

### Bugfixes


Expand Down
48 changes: 28 additions & 20 deletions docs/sources/reference/components/prometheus/prometheus.exporter.azure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,23 +66,26 @@ prometheus.exporter.azure "<LABEL>" {

You can use the following arguments with `prometheus.exporter.azure`:

| Name | Type | Description | Default | Required |
|-------------------------------| -------------- |----------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------| -------- |
| `metrics` | `list(string)` | The metrics to scrape from resources. | | yes |
| `resource_type` | `string` | The Azure Resource Type to scrape metrics for. | | yes |
| `subscriptions` | `list(string)` | List of subscriptions to scrape metrics from. | | yes |
| `azure_cloud_environment` | `string` | Name of the cloud environment to connect to. | `"azurecloud"` | no |
| `included_dimensions` | `list(string)` | List of dimensions to include on the final metrics. | | no |
| `included_resource_tags` | `list(string)` | List of resource tags to include on the final metrics. | `["owner"]` | no |
| `metric_aggregations` | `list(string)` | Aggregations to apply for the metrics produced. | | no |
| `metric_help_template` | `string` | Description of the metric. | `"Azure metric {metric} for {type} with aggregation {aggregation} as {unit}"` | no |
| `metric_name_template` | `string` | Metric template used to expose the metrics. | `"azure_{type}_{metric}_{aggregation}_{unit}"` | no |
| `metric_namespace` | `string` | Namespace for `resource_type` which have multiple levels of metrics. | | no |
| `regions` | `list(string)` | The list of regions for gathering metrics. Gathers metrics for all resources in the subscription. Can't be used if `resource_graph_query_filter` is set. | | no |
| `resource_graph_query_filter` | `string` | The [Kusto query][] filter to apply when searching for resources. Can't be used if `regions` is set. | | no |
| `timespan` | `string` | [ISO8601 Duration][] over which the metrics are being queried. | `"PT5M"` (5 minutes) | no |
| `interval` | `string` | [ISO8601 Duration][] used when to generate individual datapoints in Azure Monitor. Must be smaller than `timespan`. | `"PT1M"` (1 minute) | no |
| `validate_dimensions` | `bool` | Enable dimension validation in the azure SDK. | `false` | no |
| Name | Type | Description | Default | Required |
| ----------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | -------- |
| `metrics` | `list(string)` | The metrics to scrape from resources. | | yes |
| `resource_type` | `string` | The Azure Resource Type to scrape metrics for. | | yes |
| `subscriptions` | `list(string)` | List of subscriptions to scrape metrics from. | | yes |
| `azure_cloud_environment` | `string` | Name of the cloud environment to connect to. | `"azurecloud"` | no |
| `included_dimensions` | `list(string)` | List of dimensions to include on the final metrics. | | no |
| `included_resource_tags` | `list(string)` | List of resource tags to include on the final metrics. | `["owner"]` | no |
| `metric_aggregations` | `list(string)` | Aggregations to apply for the metrics produced. | | no |
| `metric_help_template` | `string` | Description of the metric. | `"Azure metric {metric} for {type} with aggregation {aggregation} as {unit}"` | no |
| `metric_name_template` | `string` | Metric template used to expose the metrics. | `"azure_{type}_{metric}_{aggregation}_{unit}"` | no |
| `metric_namespace` | `string` | Namespace for `resource_type` which have multiple levels of metrics. | | no |
| `regions` | `list(string)` | The list of regions for gathering metrics. Gathers metrics for all resources in the subscription. Can't be used if `resource_graph_query_filter` is set. | | no |
| `resource_graph_query_filter` | `string` | The [Kusto query][] filter to apply when searching for resources. Can't be used if `regions` is set. | | no |
| `timespan` | `string` | [ISO8601 Duration][] over which the metrics are being queried. | `"PT5M"` (5 minutes) | no |
| `interval` | `string` | [ISO8601 Duration][] used when to generate individual datapoints in Azure Monitor. Must be smaller than `timespan`. | `"PT1M"` (1 minute) | no |
| `validate_dimensions` | `bool` | Enable dimension validation in the azure SDK. | `false` | no |
| `concurrency_subscription` | `int` | Number of subscriptions that can concurrently send metric requests. | `5` | no |
| `concurrency_subscription_resource` | `int` | Number of concurrent metric requests per resource within a subscription. | `10` | no |
| `enable_caching` | `bool` | Enable internal caching to reduce redundant API calls. | `false` | no |

The list of available `resource_type` values and their corresponding `metrics` can be found in [Azure Monitor essentials][].

Expand All @@ -109,10 +112,15 @@ Valid values for `azure_cloud_environment` are `azurecloud`, `azurechinacloud`,
`validate_dimensions` is disabled by default to reduce the number of Azure exporter instances required when a `resource_type` has metrics with varying dimensions.
When `validate_dimensions` is enabled you will need one exporter instance per metric + dimension combination which is more tedious to maintain.

`timespan` and `interval` are used to control how metrics are queried from Azure Monitor.
The exporter queries metrics over the `timespan` and returns the most recent datapoint at the specified `interval`.
`timespan` and `interval` are used to control how metrics are queried from Azure Monitor.
The exporter queries metrics over the `timespan` and returns the most recent datapoint at the specified `interval`.
If you are having issues with missing metrics, try increasing the `timespan` to a larger value, such as `PT10M` for 10 minutes, or `PT15M` for 15 minutes.

The concurrency settings control how many Azure API requests can be made in parallel.
`concurrency_subscription` limits the number of subscriptions that can concurrently send metric requests, while `concurrency_subscription_resource` limits the number of concurrent metric requests per resource within a subscription.
You can adjust these values to tune performance based on your Azure subscription limits and available resources.
`enable_caching` enables internal caching to reduce redundant API calls and improve performance.

[Kusto query]: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/
[Azure Monitor essentials]: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported
[ISO8601 Duration]: https://en.wikipedia.org/wiki/ISO_8601#Durations
Expand Down Expand Up @@ -198,7 +206,7 @@ Replace the following:

`prometheus.exporter.azure` has exports that can be consumed by the following components:

- Components that consume [Targets](../../../compatibility/#targets-consumers)
* Components that consume [Targets](../../../compatibility/#targets-consumers)

{{< admonition type="note" >}}
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
Expand Down
70 changes: 40 additions & 30 deletions internal/component/prometheus/exporter/azure/azure.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,21 +26,24 @@ func createExporter(opts component.Options, args component.Arguments) (integrati
}

type Arguments struct {
Subscriptions []string `alloy:"subscriptions,attr"`
ResourceGraphQueryFilter string `alloy:"resource_graph_query_filter,attr,optional"`
ResourceType string `alloy:"resource_type,attr"`
Metrics []string `alloy:"metrics,attr"`
MetricAggregations []string `alloy:"metric_aggregations,attr,optional"`
Timespan string `alloy:"timespan,attr,optional"`
Interval string `alloy:"interval,attr,optional"`
IncludedDimensions []string `alloy:"included_dimensions,attr,optional"`
IncludedResourceTags []string `alloy:"included_resource_tags,attr,optional"`
MetricNamespace string `alloy:"metric_namespace,attr,optional"`
MetricNameTemplate string `alloy:"metric_name_template,attr,optional"`
MetricHelpTemplate string `alloy:"metric_help_template,attr,optional"`
AzureCloudEnvironment string `alloy:"azure_cloud_environment,attr,optional"`
ValidateDimensions bool `alloy:"validate_dimensions,attr,optional"`
Regions []string `alloy:"regions,attr,optional"`
Subscriptions []string `alloy:"subscriptions,attr"`
ResourceGraphQueryFilter string `alloy:"resource_graph_query_filter,attr,optional"`
ResourceType string `alloy:"resource_type,attr"`
Metrics []string `alloy:"metrics,attr"`
MetricAggregations []string `alloy:"metric_aggregations,attr,optional"`
Timespan string `alloy:"timespan,attr,optional"`
Interval string `alloy:"interval,attr,optional"`
IncludedDimensions []string `alloy:"included_dimensions,attr,optional"`
IncludedResourceTags []string `alloy:"included_resource_tags,attr,optional"`
MetricNamespace string `alloy:"metric_namespace,attr,optional"`
MetricNameTemplate string `alloy:"metric_name_template,attr,optional"`
MetricHelpTemplate string `alloy:"metric_help_template,attr,optional"`
AzureCloudEnvironment string `alloy:"azure_cloud_environment,attr,optional"`
ValidateDimensions bool `alloy:"validate_dimensions,attr,optional"`
Regions []string `alloy:"regions,attr,optional"`
ConcurrencySubscription int `alloy:"concurrency_subscription,attr,optional"`
ConcurrencySubscriptionResource int `alloy:"concurrency_subscription_resource,attr,optional"`
EnableCaching bool `alloy:"enable_caching,attr,optional"`
}

// SetToDefault implements syntax.Defaulter.
Expand All @@ -56,6 +59,10 @@ func (a *Arguments) SetToDefault() {
// to fully monitor a service which is tedious. Turning off validation eliminates this complexity. The underlying
// sdk will only give back the dimensions which are valid for particular metrics.
ValidateDimensions: false,
// Concurrency default values taken from OSS exporter
ConcurrencySubscription: 5,
ConcurrencySubscriptionResource: 10,
EnableCaching: false,
}
}

Expand All @@ -69,20 +76,23 @@ func (a *Arguments) Validate() error {

func (a *Arguments) Convert() *azure_exporter.Config {
return &azure_exporter.Config{
Subscriptions: a.Subscriptions,
ResourceGraphQueryFilter: a.ResourceGraphQueryFilter,
ResourceType: a.ResourceType,
Metrics: a.Metrics,
MetricAggregations: a.MetricAggregations,
Timespan: a.Timespan,
Interval: a.Interval,
IncludedDimensions: a.IncludedDimensions,
IncludedResourceTags: a.IncludedResourceTags,
MetricNamespace: a.MetricNamespace,
MetricNameTemplate: a.MetricNameTemplate,
MetricHelpTemplate: a.MetricHelpTemplate,
AzureCloudEnvironment: a.AzureCloudEnvironment,
ValidateDimensions: a.ValidateDimensions,
Regions: a.Regions,
Subscriptions: a.Subscriptions,
ResourceGraphQueryFilter: a.ResourceGraphQueryFilter,
ResourceType: a.ResourceType,
Metrics: a.Metrics,
MetricAggregations: a.MetricAggregations,
Timespan: a.Timespan,
Interval: a.Interval,
IncludedDimensions: a.IncludedDimensions,
IncludedResourceTags: a.IncludedResourceTags,
MetricNamespace: a.MetricNamespace,
MetricNameTemplate: a.MetricNameTemplate,
MetricHelpTemplate: a.MetricHelpTemplate,
AzureCloudEnvironment: a.AzureCloudEnvironment,
ValidateDimensions: a.ValidateDimensions,
Regions: a.Regions,
ConcurrencySubscription: a.ConcurrencySubscription,
ConcurrencySubscriptionResource: a.ConcurrencySubscriptionResource,
EnableCaching: a.EnableCaching,
}
}
13 changes: 9 additions & 4 deletions internal/static/integrations/azure_exporter/azure_exporter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,8 @@ import (
)

type Exporter struct {
cfg Config
logger *zap.SugaredLogger // used by azure client
ConcurrencyConfig azure_config.Opts
cfg Config
logger *zap.SugaredLogger // used by azure client
}

func (e Exporter) MetricsHandler() (http.Handler, error) {
Expand Down Expand Up @@ -72,7 +71,13 @@ func (e Exporter) MetricsHandler() (http.Handler, error) {
return
}

prober := metrics.NewMetricProber(ctx, logEntry, nil, settings, e.ConcurrencyConfig)
// Use concurrency settings from merged config to support query parameter overrides
concurrencyConfig := azure_config.Opts{}
concurrencyConfig.Prober.ConcurrencySubscription = mergedConfig.ConcurrencySubscription
concurrencyConfig.Prober.ConcurrencySubscriptionResource = mergedConfig.ConcurrencySubscriptionResource
concurrencyConfig.Prober.Cache = mergedConfig.EnableCaching
Copy link
Contributor

@kgeckhart kgeckhart Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep this PR to just the concurrency settings and remove the cache flag support?

This isn't the only code that would have to change to enable caching and it's a bit convoluted to support properly. I can see resource graph response caching being valuable but the metric cache feels unnecessary unless you have misconfiguration alloy to scrape more often than the timespan config.


prober := metrics.NewMetricProber(ctx, logEntry, nil, settings, concurrencyConfig)
prober.SetAzureClient(client)
prober.SetPrometheusRegistry(reg)
prober.SetAzureResourceTagManager(tagManager)
Expand Down
Loading