|
| 1 | +--- |
| 2 | +title: Deploy self-hosted gateway to Kubernetes with OpenTelemetry integration |
| 3 | +description: Learn how to deploy a self-hosted gateway component of Azure API Management on Kubernetes with OpenTelemetry |
| 4 | +author: tomkerkhove |
| 5 | + |
| 6 | +ms.service: api-management |
| 7 | +ms.workload: mobile |
| 8 | +ms.topic: article |
| 9 | +ms.author: tomkerkhove |
| 10 | +ms.date: 12/17/2021 |
| 11 | +--- |
| 12 | + |
| 13 | +# Deploy self-hosted gateway to Kubernetes with OpenTelemetry integration |
| 14 | + |
| 15 | +This article describes the steps for deploying the self-hosted gateway component of Azure API Management to a Kubernetes cluster and automatically send all metrics to an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/). |
| 16 | + |
| 17 | +[!INCLUDE [preview](./includes/preview/preview-callout-self-hosted-gateway-opentelemetry.md)] |
| 18 | + |
| 19 | +You learn how to: |
| 20 | + |
| 21 | +> [!div class="checklist"] |
| 22 | +> * Configure and deploy a standalone OpenTelemetry Collector on Kubernetes |
| 23 | +> * Deploy the self-hosted gateway with OpenTelemetry metrics. |
| 24 | +> * Generate metrics by consuming APIs on the self-hosted gateway. |
| 25 | +> * Use the metrics from the OpenTelemetry Collector. |
| 26 | +
|
| 27 | +## Prerequisites |
| 28 | + |
| 29 | +- [Create an Azure API Management instance](get-started-create-service-instance.md) |
| 30 | +- [Create an Azure Kubernetes cluster](../aks/kubernetes-walkthrough-portal.md) |
| 31 | +- [Provision a self-hosted gateway resource in your API Management instance](api-management-howto-provision-self-hosted-gateway.md). |
| 32 | + |
| 33 | + |
| 34 | +## Introduction to OpenTelemetry |
| 35 | + |
| 36 | +[OpenTelemetry](https://opentelemetry.io/) is a set of open-source tools and frameworks for logging, metrics, and tracing in a vendor-neutral way. |
| 37 | + |
| 38 | +[!INCLUDE [preview](./includes/preview/preview-callout-self-hosted-gateway-opentelemetry.md)] |
| 39 | + |
| 40 | +The self-hosted gateway can be configured to automatically collect and send metrics to an [OpenTelemetry Collector](https://opentelemetry.io/docs/concepts/components/#collector). This allows you to bring your own metrics collection and reporting solution for the self-hosted gateway. |
| 41 | + |
| 42 | +> [!NOTE] |
| 43 | +> OpenTelemetry is an incubating project of the [Cloud Native Computing Foundation (CNCF) ecosystem](https://www.cncf.io/). |
| 44 | +
|
| 45 | +### Metrics |
| 46 | + |
| 47 | +The self-hosted gateway will automatically start measuring the following metrics: |
| 48 | + |
| 49 | +- Requests |
| 50 | +- DurationInMs |
| 51 | +- BackendDurationInMs |
| 52 | +- ClientDurationInMs |
| 53 | +- GatewayDurationInMs |
| 54 | + |
| 55 | +They are automatically exported to the configured OpenTelemetry Collector every 1 minute with additional dimensions. |
| 56 | + |
| 57 | +## Deploy the OpenTelemetry Collector |
| 58 | + |
| 59 | +We will start by deploying a standalone OpenTelemetry Collector on Kubernetes by using Helm. |
| 60 | + |
| 61 | +> [!TIP] |
| 62 | +> While we will be using the Collector Helm chart, they also provide an [OpenTelemetry Collector Operator](https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-operator) |
| 63 | +
|
| 64 | +To start with, we have to add the Helm chart repository: |
| 65 | +1. Add the Helm repository |
| 66 | + |
| 67 | + ```console |
| 68 | + helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts |
| 69 | + ``` |
| 70 | + |
| 71 | +2. Update repo to fetch the latest Helm charts. |
| 72 | + |
| 73 | + ```console |
| 74 | + helm repo update |
| 75 | + ``` |
| 76 | + |
| 77 | +3. Verify your Helm configuration by listing all available charts. |
| 78 | + |
| 79 | + ```console |
| 80 | + $ helm search repo open-telemetry |
| 81 | + NAME CHART VERSION APP VERSION DESCRIPTION |
| 82 | + open-telemetry/opentelemetry-collector 0.8.1 0.37.1 OpenTelemetry Collector Helm chart for Kubernetes |
| 83 | + open-telemetry/opentelemetry-operator 0.4.0 0.37.0 OpenTelemetry Operator Helm chart for Kubernetes |
| 84 | + ``` |
| 85 | + |
| 86 | +Now that we have the chart repository configured, we can deploy the OpenTelemetry Collector to our cluster: |
| 87 | + |
| 88 | +1. Create a local configuration file called `opentelemetry-collector-config.yml` with the following configuration: |
| 89 | + |
| 90 | + ```yaml |
| 91 | + agentCollector: |
| 92 | + enabled: false |
| 93 | + standaloneCollector: |
| 94 | + enabled: true |
| 95 | + config: |
| 96 | + exporters: |
| 97 | + prometheus: |
| 98 | + endpoint: "0.0.0.0:8889" |
| 99 | + namespace: azure_apim |
| 100 | + send_timestamps: true |
| 101 | + service: |
| 102 | + pipelines: |
| 103 | + metrics: |
| 104 | + exporters: |
| 105 | + - prometheus |
| 106 | + service: |
| 107 | + type: LoadBalancer |
| 108 | + ports: |
| 109 | + jaeger-compact: |
| 110 | + enabled: false |
| 111 | + prom-exporter: |
| 112 | + enabled: true |
| 113 | + containerPort: 8889 |
| 114 | + servicePort: 8889 |
| 115 | + protocol: TCP |
| 116 | + ``` |
| 117 | +
|
| 118 | +This allows us to use a standalone collector with the Prometheus exporter being exposed on port `8889`. To expose the Prometheus metrics, we are asking the Helm chart to configure a ´LoadBalancer` service. |
| 119 | + |
| 120 | +> [!NOTE] |
| 121 | +> We are disabling the compact Jaeger port given it uses UDP and `LoadBalancer` service does not allow you to have multiple protocols at the same time. |
| 122 | + |
| 123 | +2. Install the Helm chart with our configuration: |
| 124 | + |
| 125 | + ```console |
| 126 | + helm install opentelemetry-collector open-telemetry/opentelemetry-collector --values .\opentelemetry-collector-config.yml |
| 127 | + ``` |
| 128 | + |
| 129 | +3. Verify the installation by getting all the resources for our Helm chart |
| 130 | + |
| 131 | + ```console |
| 132 | + $ kubectl get all -l app.kubernetes.io/instance=opentelemetry-collector |
| 133 | + NAME READY STATUS RESTARTS AGE |
| 134 | + pod/opentelemetry-collector-58477c8c89-dstwd 1/1 Running 0 27m |
| 135 | +
|
| 136 | + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE |
| 137 | + service/opentelemetry-collector LoadBalancer 10.0.175.135 20.103.18.53 14250:30982/TCP,14268:32461/TCP,4317:31539/TCP,4318:31581/TCP,8889:32420/TCP,9411:30003/TCP 27m |
| 138 | +
|
| 139 | + NAME READY UP-TO-DATE AVAILABLE AGE |
| 140 | + deployment.apps/opentelemetry-collector 1/1 1 1 27m |
| 141 | +
|
| 142 | + NAME DESIRED CURRENT READY AGE |
| 143 | + replicaset.apps/opentelemetry-collector-58477c8c89 1 1 1 27m |
| 144 | + ``` |
| 145 | + |
| 146 | +4. Take note of the external IP of the service, so we can query it later on. |
| 147 | + |
| 148 | +With our OpenTelemetry Collector installed, we can now deploy the self-hosted gateway to our cluster. |
| 149 | + |
| 150 | +## Deploy the self-hosted gateway |
| 151 | + |
| 152 | +> [!IMPORTANT] |
| 153 | +> For a detailed overview on how to deploy the self-hosted gateway with Helm and how to get the required configuration, we recommend reading [this article](how-to-deploy-self-hosted-gateway-kubernetes-helm.md). |
| 154 | + |
| 155 | +In this section, we will deploy the self-hosted gateway to our cluster with Helm and configure it to send OpenTelemetry metrics to the OpenTelemetry Collector. |
| 156 | + |
| 157 | +1. Install the Helm chart and configure it to use OpenTelemetry metrics: |
| 158 | + |
| 159 | + ```console |
| 160 | + helm install azure-api-management-gateway \ |
| 161 | + --set gateway.configuration.uri='<your configuration url>' \ |
| 162 | + --set gateway.auth.key='<your auth token>' \ |
| 163 | + --set observability.opentelemetry.enabled=true \ |
| 164 | + --set observability.opentelemetry.collector.uri=http://opentelemetry-collector:4317 \ |
| 165 | + --set service.type=LoadBalancer \ |
| 166 | + azure-apim-gateway/azure-api-management-gateway |
| 167 | + ``` |
| 168 | + |
| 169 | +> [!NOTE] |
| 170 | +> `opentelemetry-collector` in the command above is the name of the OpenTelemetry Collector. Update the name if your service has a different name. |
| 171 | + |
| 172 | +2. Verify the installation by getting all the resources for our Helm chart |
| 173 | + |
| 174 | + ```console |
| 175 | + $ kubectl get all -l app.kubernetes.io/instance=apim-gateway |
| 176 | + NAME READY STATUS RESTARTS AGE |
| 177 | + pod/apim-gateway-azure-api-management-gateway-fb77c6d49-rffwq 1/1 Running 0 63m |
| 178 | +
|
| 179 | + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE |
| 180 | + service/apim-gateway-azure-api-management-gateway LoadBalancer 10.0.67.177 20.71.82.110 8080:32267/TCP,8081:32065/TCP 63m |
| 181 | +
|
| 182 | + NAME READY UP-TO-DATE AVAILABLE AGE |
| 183 | + deployment.apps/apim-gateway-azure-api-management-gateway 1/1 1 1 63m |
| 184 | +
|
| 185 | + NAME DESIRED CURRENT READY AGE |
| 186 | + replicaset.apps/apim-gateway-azure-api-management-gateway-fb77c6d49 1 1 1 63m |
| 187 | + ``` |
| 188 | + |
| 189 | +3. Take note of the external IP of the self-hosted gateway's service, so we can query it later on. |
| 190 | + |
| 191 | +## Generate and consume the OpenTelemetry metrics |
| 192 | + |
| 193 | +Now that both our OpenTelemetry Collector and the self-hosted gateway are deployed, we can start consuming the APIs to generate metrics. |
| 194 | + |
| 195 | +> [!NOTE] |
| 196 | +> We will be consuming the default "Echo API" for this walkthrough. |
| 197 | +> |
| 198 | +> Make sure that it is configured to: |
| 199 | +> - Allow HTTP requests |
| 200 | +> - Allow your self-hosted gateway to expose it |
| 201 | + |
| 202 | +1. Query the Echo API in the self-hosted gateway: |
| 203 | + |
| 204 | + ```console |
| 205 | + $ curl -i "http://<self-hosted-gateway-ip>:8080/echo/resource?param1=sample&subscription-key=abcdef0123456789" |
| 206 | + HTTP/1.1 200 OK |
| 207 | + Date: Mon, 20 Dec 2021 12:58:09 GMT |
| 208 | + Server: Microsoft-IIS/8.5 |
| 209 | + Content-Length: 0 |
| 210 | + Cache-Control: no-cache |
| 211 | + Pragma: no-cache |
| 212 | + Expires: -1 |
| 213 | + Accept: */* |
| 214 | + Host: echoapi.cloudapp.net |
| 215 | + User-Agent: curl/7.68.0 |
| 216 | + X-Forwarded-For: 10.244.1.1 |
| 217 | + traceparent: 00-3192030c89fd7a60ef4c9749d6bdef0c-f4eeeee46f770061-01 |
| 218 | + Request-Id: |3192030c89fd7a60ef4c9749d6bdef0c.f4eeeee46f770061. |
| 219 | + Request-Context: appId=cid-v1:c24f5e00-aa25-47f2-bbb5-035847e7f52a |
| 220 | + X-Powered-By: Azure API Management - http://api.azure.com/,ASP.NET |
| 221 | + X-AspNet-Version: 4.0.30319 |
| 222 | + ``` |
| 223 | + |
| 224 | +The self-hosted gateway will now measure the request and send the metrics to the OpenTelemetry Collector. |
| 225 | + |
| 226 | +2. Query Prometheus endpoint on collector on `http://<collector-service-ip>:8889/metrics`. You should see metrics similar to the following: |
| 227 | + |
| 228 | + ```raw |
| 229 | + # HELP azure_apim_BackendDurationInMs |
| 230 | + # TYPE azure_apim_BackendDurationInMs histogram |
| 231 | + azure_apim_BackendDurationInMs_bucket{Hostname="20.71.82.110",le="5"} 0 1640093731340 |
| 232 | + [...] |
| 233 | + azure_apim_BackendDurationInMs_count{Hostname="20.71.82.110"} 22 1640093731340 |
| 234 | + # HELP azure_apim_ClientDurationInMs |
| 235 | + # TYPE azure_apim_ClientDurationInMs histogram |
| 236 | + azure_apim_ClientDurationInMs_bucket{Hostname="20.71.82.110",le="5"} 22 1640093731340 |
| 237 | + [...] |
| 238 | + azure_apim_ClientDurationInMs_count{Hostname="20.71.82.110"} 22 1640093731340 |
| 239 | + # HELP azure_apim_DurationInMs |
| 240 | + # TYPE azure_apim_DurationInMs histogram |
| 241 | + azure_apim_DurationInMs_bucket{Hostname="20.71.82.110",le="5"} 0 1640093731340 |
| 242 | + [...] |
| 243 | + azure_apim_DurationInMs_count{Hostname="20.71.82.110"} 22 1640093731340 |
| 244 | + # HELP azure_apim_GatewayDurationInMs |
| 245 | + # TYPE azure_apim_GatewayDurationInMs histogram |
| 246 | + azure_apim_GatewayDurationInMs_bucket{Hostname="20.71.82.110",le="5"} 0 1640093731340 |
| 247 | + [...] |
| 248 | + azure_apim_GatewayDurationInMs_count{Hostname="20.71.82.110"} 22 1640093731340 |
| 249 | + # HELP azure_apim_Requests |
| 250 | + # TYPE azure_apim_Requests counter |
| 251 | + azure_apim_Requests{BackendResponseCode="200",BackendResponseCodeCategory="2xx",Cache="None",GatewayId="Docs",Hostname="20.71.82.110",LastErrorReason="None",Location="GitHub",ResponseCode="200",ResponseCodeCategory="2xx",Status="Successful"} 22 1640093731340 |
| 252 | + ``` |
| 253 | + |
| 254 | +## Cleaning up |
| 255 | + |
| 256 | +Now that the tutorial is over, you can easily clean up your cluster as following: |
| 257 | + |
| 258 | +1. Uninstall the self-hosted gateway Helm chart: |
| 259 | + |
| 260 | + ```console |
| 261 | + helm uninstall apim-gateway |
| 262 | + ``` |
| 263 | + |
| 264 | +2. Uninstall the OpenTelemetry Collector: |
| 265 | + |
| 266 | + ```console |
| 267 | + helm uninstall opentelemetry-collector |
| 268 | + ``` |
| 269 | + |
| 270 | +## Next steps |
| 271 | + |
| 272 | +- To learn more about the self-hosted gateway, see [Self-hosted gateway overview](self-hosted-gateway-overview.md). |
| 273 | +* To learn more about the [observability capabilities of the Azure API Management gateways](observability.md). |
| 274 | +- [Deploy self-hosted gateway to Azure Arc-enabled Kubernetes cluster](how-to-deploy-self-hosted-gateway-azure-arc.md) |
0 commit comments