Prometheus Metrics Exporting to Azure Monitor

Prometheus To Azure Monitor

A Service to export queryable Prometheus Metrics to Azure Monitor for aggregation, persistence, and alerting.

Sample Jackson exposed Prometheus Metrics.

Example Service:

Prometheus Exporter for Azure Monitor

A solution built for a customer (Inside Sales) by Devdatta Waghdhare and Vinicius Souza. Permission has been given to open source this into a generic service/library. We need to look into steps needed to audit this first. Where can we find resources for this? Are there other tools?:

• https://docs.opensource.microsoft.com/tools/cg.html
• https://docs.opensource.microsoft.com/
• https://opensource.microsoft.com/resources/release
• https://opensource.microsoft.com/
• https://microsoft.sharepoint.com/teams/msopensource/Pages/default.aspx
• https://cloudblogs.microsoft.com/opensource/
• https://docs.opensource.microsoft.com/
• https://microsoft.sharepoint.com/teams/OpenSourceMicrosoft
• Adjacent Teams discussion

Steps to deploy this service to a cluster with Prometheus running:

• Requirements:
  • Active configured Prometheus Service
  • Azure RBAC role with Contributor and Monitoring Metrics Publisher roles for targeted Resource(eg. AzureCluster)
• Steps:
  • Configure Prometheus URL, custom metric namespace and queried metrics in values.yaml and cronjob schedule.
  • In deployment directory, deploy via script:
    • sh install_helm_chart.sh -a <AzureResourceID> -b <AzureADClientID> -c <AzureADClientSecret> -d <AzureADTenantID> -e <AzureMonitorRegion> -f <tag> -g <AksNamespace> -h <PromToAzureMonitorCMD> -i <RepoImageName> -n <ChartName> -m <helmLog:Boolean> -o <deleteBeforeUpdate:Boolean>
    • <AzureResourceID>
      • Full resource ID
        • az aks show --resource-group <resource group> --name <cluster name> --query "id" --output table
      • /subscriptions/<SubscriptionID>/resourcegroups/<resourcegroup>/providers/Microsoft.ContainerService/managedClusters/<ClusterName>
    • <AzureADClientID>
      • Azure AD Application ID of RBAC role
      • <ApplicationID>
    • <AzureADClientSecret>
      • Azure AD Application Key
        • Settings -> Keys -> Create password/key and get value
      • <password key value string>
    • <AzureADTenantID>
      • Aka Directory ID
        • In portal: Azure Active Directory -> Properties -> Directory ID
      • <DirectoryID>
    • <AzureMonitorRegion>
      • Region where the resource exists. Must be a Azure Monitor supported region
      • eg. westus2
    • <tag>
      • Docker image tag name
      • eg. v0.1
    • <AksNamespace>
      • Kubernetes Name space for this service
      • eg. prom2azure
    • <PromToAzureMonitorCMD>
      • Command to run with cron job
      • ie. pull or push
    • <RepoImageName>
      • Docker/Azure container repository and image name
      • eg. mtarng/prom2azure
    • <ChartName>
      • Helm chart name
      • eg. prom2azurechart
    • <helmLog:Boolean>
      • Helm install debug dry-run
      • true or false
    • <deleteBeforeUpdate:Boolean>
      • Helm delete before update
      • true or false

Azure Monitor

Custom Metrics in Azure Monitor

Current limitations as defined by Documentation

REST API Documentation

Supported Regions

Quotas and Limits

Category	Limit
Active time series/subscriptions/region*	50,000
Dimension keys per metric	10
String length for metric namespaces, metric names, dimension keys, and dimension values	256 characters

*An active time series is defined as any unique combination of metric, dimension key, or dimension value that has had metric values published in the past 12 hours.