|
| 1 | +--- |
| 2 | +title: Customize the application routing add-on for Azure Kubernetes Service (AKS) |
| 3 | +description: Understand what advanced configuration options are supported with the application routing add-on for Azure Kubernetes Service. |
| 4 | +ms.subservice: aks-networking |
| 5 | +ms.custom: devx-track-azurecli |
| 6 | +ms.topic: how-to |
| 7 | +ms.date: 11/03/2023 |
| 8 | +--- |
| 9 | + |
| 10 | +# Advanced Ingress configurations with the application routing add-on |
| 11 | + |
| 12 | +An Ingress is an API object that defines rules, which allow external access to services in an Azure Kubernetes Service (AKS) cluster. When you create an Ingress object that uses the application routing add-on nginx Ingress classes, the add-on creates, configures, and manages one or more Ingress controllers in your AKS cluster. |
| 13 | + |
| 14 | +This article shows you how to set up an advanced Ingress configuration to encrypt the traffic and use Azure DNS to manage DNS zones. |
| 15 | + |
| 16 | +## Application routing add-on with nginx features |
| 17 | + |
| 18 | +The application routing add-on with nginx delivers the following: |
| 19 | + |
| 20 | +* Easy configuration of managed nginx Ingress controllers based on [Kubernetes nginx Ingress controller][kubernetes-nginx-ingress]. |
| 21 | +* Integration with an external DNS such as [Azure DNS][azure-dns-overview] for public and private zone management |
| 22 | +* SSL termination with certificates stored in a key vault, such as [Azure Key Vault][azure-key-vault-overview]. |
| 23 | + |
| 24 | +## Prerequisites |
| 25 | + |
| 26 | +- An AKS cluster with the [application routing add-on][app-routing-add-on-basic-configuration]. |
| 27 | +- Azure Key Vault if you want to configure SSL termination and store certificates in the vault hosted in Azure. |
| 28 | +- Azure DNS if you want to configure public and private zone management and host them in Azure. |
| 29 | + |
| 30 | +## Connect to your AKS cluster |
| 31 | + |
| 32 | +To connect to the Kubernetes cluster from your local computer, you use `kubectl`, the Kubernetes command-line client. You can install it locally using the [az aks install-cli][az-aks-install-cli] command. If you use the Azure Cloud Shell, `kubectl` is already installed. |
| 33 | + |
| 34 | +Configure kubectl to connect to your Kubernetes cluster using the [`az aks get-credentials`][az-aks-get-credentials] command. |
| 35 | + |
| 36 | +```bash |
| 37 | +az aks get-credentials -g <ResourceGroupName> -n <ClusterName> |
| 38 | +``` |
| 39 | + |
| 40 | +## Terminate HTTPS traffic |
| 41 | + |
| 42 | +To enable support for HTTPS traffic, see the following prerequisites: |
| 43 | + |
| 44 | +* **azure-keyvault-secrets-provider**: The [Secret Store CSI provider][secret-store-csi-provider] for Azure Key Vault is required to retrieve the certificates from Azure Key Vault. |
| 45 | + |
| 46 | + > [!IMPORTANT] |
| 47 | + > To enable the add-on to reload certificates from Azure Key Vault when they change, you should to enable the [secret autorotation feature][csi-secrets-store-autorotation] of the Secret Store CSI driver with the `--enable-secret-rotation` argument. When autorotation is enabled, the driver updates the pod mount and the Kubernetes secret by polling for changes periodically, based on the rotation poll interval you define. The default rotation poll interval is two minutes. |
| 48 | +
|
| 49 | +* An SSL certificate. If you don't have one, you can [create a certificate][create-and-export-a-self-signed-ssl-certificate]. |
| 50 | + |
| 51 | +### Enable key vault secrets provider |
| 52 | + |
| 53 | +To enable application routing on your cluster, use the [`az aks enable-addons`][az-aks-enable-addons] command specifying `azure-keyvault-secrets-provider` with the `--addons` argument and the `--enable-secret-rotation` argument. |
| 54 | + |
| 55 | +```azurecli-interactive |
| 56 | +az aks enable-addons -g <ResourceGroupName> -n <ClusterName> --addons azure-keyvault-secrets-provider --enable-secret-rotation |
| 57 | +``` |
| 58 | + |
| 59 | +### Create an Azure Key Vault to store the certificate |
| 60 | + |
| 61 | +> [!NOTE] |
| 62 | +> If you already have an Azure Key Vault, you can skip this step. |
| 63 | +
|
| 64 | +Create an Azure Key Vault using the [`az keyvault create`][az-keyvault-create] command. |
| 65 | + |
| 66 | +```azurecli-interactive |
| 67 | +az keyvault create -g <ResourceGroupName> -l <Location> -n <KeyVaultName> |
| 68 | +``` |
| 69 | + |
| 70 | +### Create and export a self-signed SSL certificate |
| 71 | + |
| 72 | +1. Create a self-signed SSL certificate to use with the Ingress using the `openssl req` command. Make sure you replace *`<Hostname>`* with the DNS name you're using. |
| 73 | + |
| 74 | + ```bash |
| 75 | + openssl req -new -x509 -nodes -out aks-ingress-tls.crt -keyout aks-ingress-tls.key -subj "/CN=<Hostname>" -addext "subjectAltName=DNS:<Hostname>" |
| 76 | + ``` |
| 77 | + |
| 78 | +2. Export the SSL certificate and skip the password prompt using the `openssl pkcs12 -export` command. |
| 79 | + |
| 80 | + ```bash |
| 81 | + openssl pkcs12 -export -in aks-ingress-tls.crt -inkey aks-ingress-tls.key -out aks-ingress-tls.pfx |
| 82 | + ``` |
| 83 | + |
| 84 | +### Import certificate into Azure Key Vault |
| 85 | + |
| 86 | +Import the SSL certificate into Azure Key Vault using the [`az keyvault certificate import`][az-keyvault-certificate-import] command. If your certificate is password protected, you can pass the password through the `--password` flag. |
| 87 | + |
| 88 | +```azurecli-interactive |
| 89 | +az keyvault certificate import --vault-name <KeyVaultName> -n <KeyVaultCertificateName> -f aks-ingress-tls.pfx [--password <certificate password if specified>] |
| 90 | +``` |
| 91 | +
|
| 92 | +### Retrieve the add-on's managed identity object ID |
| 93 | +
|
| 94 | +You use the managed identity in the next steps to grant permissions to manage the Azure DNS zone and retrieve secrets and certificates from the Azure Key Vault. |
| 95 | +
|
| 96 | +Get the add-on's managed identity object ID using the [`az aks show`][az-aks-show] command and setting the output to a variable named `MANAGEDIDENTITY_OBJECTID`. |
| 97 | +
|
| 98 | +```bash |
| 99 | +# Provide values for your environment |
| 100 | +RGNAME=<ResourceGroupName> |
| 101 | +CLUSTERNAME=<ClusterName> |
| 102 | +MANAGEDIDENTITY_OBJECTID=$(az aks show -g ${RGNAME} -n ${CLUSTERNAME} --query ingressProfile.webAppRouting.identity.objectId -o tsv) |
| 103 | +``` |
| 104 | +
|
| 105 | +### Grant the add-on permissions to retrieve certificates from Azure Key Vault |
| 106 | +
|
| 107 | +The application routing add-on creates a user-created managed identity in the cluster resource group. You need to grant permissions to the managed identity so it can retrieve SSL certificates from the Azure Key Vault. |
| 108 | +
|
| 109 | +Azure Key Vault offers [two authorization systems][authorization-systems]: **Azure role-based access control (Azure RBAC)**, which operates on the management plane, and the **access policy model**, which operates on both the management plane and the data plane. To find out which system your key vault is using, you can query the `enableRbacAuthorization` property. |
| 110 | +
|
| 111 | +```azurecli-interactive |
| 112 | +az keyvault show --name <KeyVaultName> --query properties.enableRbacAuthorization |
| 113 | +``` |
| 114 | +
|
| 115 | +If Azure RBAC authorization is enabled for your key vault, you should configure permissions using Azure RBAC. Add the `Key Vault Secrets User` role assignment to the key vault by running the following commands. |
| 116 | +
|
| 117 | +```azurecli-interactive |
| 118 | +KEYVAULTID=$(az keyvault show --name <KeyVaultName> --query "id" --output tsv) |
| 119 | +az role assignment create --role "Key Vault Secrets User" --assignee $MANAGEDIDENTITY_OBJECTID --scope $KEYVAULTID |
| 120 | +``` |
| 121 | +
|
| 122 | +If Azure RBAC authorization isn't enabled for your key vault, you should configure permissions using the access policy model. Grant `GET` permissions for the application routing add-on to retrieve certificates from Azure Key Vault using the [`az keyvault set-policy`][az-keyvault-set-policy] command. |
| 123 | +
|
| 124 | +```azurecli-interactive |
| 125 | +az keyvault set-policy --name <KeyVaultName> --object-id $MANAGEDIDENTITY_OBJECTID --secret-permissions get --certificate-permissions get |
| 126 | +``` |
| 127 | +
|
| 128 | +## Configure the add-on to use Azure DNS to manage DNS zones |
| 129 | +
|
| 130 | +To enable support for DNS zones, see the following prerequisites: |
| 131 | +
|
| 132 | +* The app routing add-on can be configured to automatically create records on one or more Azure public and private DNS zones for hosts defined on Ingress resources. All global Azure DNS zones need to be in the same resource group, and all private Azure DNS zones need to be in the same resource group. If you don't have an Azure DNS zone, you can [create one][create-an-azure-dns-zone]. |
| 133 | +
|
| 134 | + > [!NOTE] |
| 135 | + > If you plan to use Azure DNS, you need to update the add-on to include the `--dns-zone-resource-ids` argument. You can pass a comma separated list of multiple public or private Azure DNS zone resource IDs. |
| 136 | +
|
| 137 | +### Create a global Azure DNS zone |
| 138 | +
|
| 139 | +1. Create an Azure DNS zone using the [`az network dns zone create`][az-network-dns-zone-create] command. |
| 140 | +
|
| 141 | + ```azurecli-interactive |
| 142 | + az network dns zone create -g <ResourceGroupName> -n <ZoneName> |
| 143 | + ``` |
| 144 | +
|
| 145 | +1. Retrieve the resource ID for the DNS zone using the [`az network dns zone show`][az-network-dns-zone-show] command and set the output to a variable named *ZONEID*. |
| 146 | +
|
| 147 | + ```azurecli-interactive |
| 148 | + ZONEID=$(az network dns zone show -g <ResourceGroupName> -n <ZoneName> --query "id" --output tsv) |
| 149 | + ``` |
| 150 | +
|
| 151 | +1. Grant **DNS Zone Contributor** permissions on the DNS zone using the [`az role assignment create`][az-role-assignment-create] command. |
| 152 | +
|
| 153 | + ```azurecli-interactive |
| 154 | + az role assignment create --role "DNS Zone Contributor" --assignee $MANAGEDIDENTITY_OBJECTID --scope $ZONEID |
| 155 | + ``` |
| 156 | +
|
| 157 | +1. Update the add-on to enable the integration with Azure DNS and install the **external-dns** controller using the [`az aks addon update`][az-aks-addon-update] command. |
| 158 | +
|
| 159 | + ```azurecli-interactive |
| 160 | + az aks addon update -g <ResourceGroupName> -n <ClusterName> --addon web_application_routing --dns-zone-resource-ids=$ZONEID |
| 161 | + ``` |
| 162 | +
|
| 163 | +## Create the Ingress |
| 164 | +
|
| 165 | +The application routing add-on creates an Ingress class on the cluster named *webapprouting.kubernetes.azure.com*. When you create an Ingress object with this class, it activates the add-on. |
| 166 | +
|
| 167 | +1. Get the certificate URI to use in the Ingress from Azure Key Vault using the [`az keyvault certificate show`][az-keyvault-certificate-show] command. |
| 168 | +
|
| 169 | + ```azurecli-interactive |
| 170 | + az keyvault certificate show --vault-name <KeyVaultName> -n <KeyVaultCertificateName> --query "id" --output tsv |
| 171 | + ``` |
| 172 | +
|
| 173 | +2. Copy the following YAML manifest into a new file named **ingress.yaml** and save the file to your local computer. |
| 174 | +
|
| 175 | + > [!NOTE] |
| 176 | + > Update *`<Hostname>`* with your DNS host name and *`<KeyVaultCertificateUri>`* with the ID returned from Azure Key Vault. |
| 177 | + > The *`secretName`* key in the `tls` section defines the name of the secret that contains the certificate for this Ingress resource. This certificate will be presented in the browser when a client browses to the URL defined in the `<Hostname>` key. Make sure that the value of `secretName` is equal to `keyvault-` followed by the value of the Ingress resource name (from `metadata.name`). In the example YAML, secretName will need to be equal to `keyvault-<your Ingress name>`. |
| 178 | +
|
| 179 | + ```yml |
| 180 | + apiVersion: networking.k8s.io/v1 |
| 181 | + kind: Ingress |
| 182 | + metadata: |
| 183 | + annotations: |
| 184 | + kubernetes.azure.com/tls-cert-keyvault-uri: <KeyVaultCertificateUri> |
| 185 | + name: aks-helloworld |
| 186 | + namespace: hello-web-app-routing |
| 187 | + spec: |
| 188 | + ingressClassName: webapprouting.kubernetes.azure.com |
| 189 | + rules: |
| 190 | + - host: <Hostname> |
| 191 | + http: |
| 192 | + paths: |
| 193 | + - backend: |
| 194 | + service: |
| 195 | + name: aks-helloworld |
| 196 | + port: |
| 197 | + number: 80 |
| 198 | + path: / |
| 199 | + pathType: Prefix |
| 200 | + tls: |
| 201 | + - hosts: |
| 202 | + - <Hostname> |
| 203 | + secretName: keyvault-<your ingress name> |
| 204 | + ``` |
| 205 | +
|
| 206 | +3. Create the cluster resources using the [`kubectl apply`][kubectl-apply] command. |
| 207 | +
|
| 208 | + ```bash |
| 209 | + kubectl apply -f ingress.yaml -n hello-web-app-routing |
| 210 | + ``` |
| 211 | +
|
| 212 | + The following example output shows the created resource: |
| 213 | +
|
| 214 | + ```output |
| 215 | + Ingress.networking.k8s.io/aks-helloworld created |
| 216 | + ``` |
| 217 | +
|
| 218 | +## Verify the managed Ingress was created |
| 219 | +
|
| 220 | +You can verify the managed Ingress was created using the [`kubectl get ingress`][kubectl-get] command. |
| 221 | +
|
| 222 | +```bash |
| 223 | +kubectl get ingress -n hello-web-app-routing |
| 224 | +``` |
| 225 | +
|
| 226 | +The following example output shows the created managed Ingress: |
| 227 | +
|
| 228 | +```output |
| 229 | +NAME CLASS HOSTS ADDRESS PORTS AGE |
| 230 | +aks-helloworld webapprouting.kubernetes.azure.com myapp.contoso.com 20.51.92.19 80, 443 4m |
| 231 | +``` |
| 232 | +
|
| 233 | +## Next steps |
| 234 | +
|
| 235 | +Learn about monitoring the Ingress-nginx controller metrics included with the application routing add-on with [with Prometheus in Grafana][prometheus-in-grafana] (preview) as part of analyzing the performance and usage of your application. |
| 236 | +
|
| 237 | +<!-- LINKS - external --> |
| 238 | +[kubectl-apply]: https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply |
| 239 | +[kubectl-get]: https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#get |
| 240 | +
|
| 241 | +<!-- LINKS - internal --> |
| 242 | +[app-routing-add-on-basic-configuration]: app-routing.md |
| 243 | +[secret-store-csi-provider]: csi-secrets-store-driver.md |
| 244 | +[csi-secrets-store-autorotation]: csi-secrets-store-configuration-options.md#enable-and-disable-auto-rotation |
| 245 | +[az-keyvault-set-policy]: /cli/azure/keyvault#az-keyvault-set-policy |
| 246 | +[azure-key-vault-overview]: ../key-vault/general/overview.md |
| 247 | +[az-aks-addon-update]: /cli/azure/aks/addon#az-aks-addon-update |
| 248 | +[az-network-dns-zone-show]: /cli/azure/network/dns/zone#az-network-dns-zone-show |
| 249 | +[az-role-assignment-create]: /cli/azure/role/assignment#az-role-assignment-create |
| 250 | +[az-network-dns-zone-create]: /cli/azure/network/dns/zone#az-network-dns-zone-create |
| 251 | +[az-keyvault-certificate-import]: /cli/azure/keyvault/certificate#az-keyvault-certificate-import |
| 252 | +[az-keyvault-create]: /cli/azure/keyvault#az-keyvault-create |
| 253 | +[authorization-systems]: ../key-vault/general/rbac-access-policy.md |
| 254 | +[az-aks-install-cli]: /cli/azure/aks#az-aks-install-cli |
| 255 | +[az-aks-get-credentials]: /cli/azure/aks#az-aks-get-credentials |
| 256 | +[create-and-export-a-self-signed-ssl-certificate]: #create-and-export-a-self-signed-ssl-certificate |
| 257 | +[create-an-azure-dns-zone]: #create-a-global-azure-dns-zone |
| 258 | +[azure-dns-overview]: ../dns/dns-overview.md |
| 259 | +[az-keyvault-certificate-show]: /cli/azure/keyvault/certificate#az-keyvault-certificate-show |
| 260 | +[az-aks-enable-addons]: /cli/azure/aks/addon#az-aks-enable-addon |
| 261 | +[az-aks-show]: /cli/azure/aks/addon#az-aks-show |
| 262 | +[prometheus-in-grafana]: app-routing-nginx-prometheus.md |
0 commit comments