Skip to content

Commit 554a784

Browse files
sethmanheimsethmanheim
authored
Merge pull request #16291 from sethmanheim/wk-id1108
AKS Arc workload identity article
2 parents 51ad12a + 6af824b commit 554a784

File tree

2 files changed

+320
-0
lines changed

2 files changed

+320
-0
lines changed

‎AKS-Hybrid/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,8 @@
8282
href: retrieve-admin-kubeconfig.md
8383
- name: Restrict SSH access
8484
href: restrict-ssh-access.md
85+
- name: Deploy and configure Workload Identity
86+
href: workload-identity.md
8587
- name: Storage
8688
items:
8789
- name: CSI storage drivers

‎AKS-Hybrid/workload-identity.md

Lines changed: 318 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,318 @@
1+
---
2+
title: Deploy and configure Workload Identity on an AKS enabled by Azure Arc cluster (preview)
3+
description: Learn how to deploy and configure an AKS Arc cluster with workload identity.
4+
author: sethmanheim
5+
ms.author: sethm
6+
ms.topic: how-to
7+
ms.date: 11/08/2024
8+
9+
---
10+
11+
# Deploy and configure Workload Identity on an AKS enabled by Azure Arc cluster (preview)
12+
13+
[!INCLUDE [hci-applies-to-23h2](includes/hci-applies-to-23h2.md)]
14+
15+
Workload identity federation allows you to configure a user-assigned managed identity or app registration in Microsoft Entra ID to trust tokens from an external identity provider (IdP), such as Kubernetes, enabling access to resources protected by Microsoft Entra, like Azure Key Vault or Azure Blob storage.
16+
17+
Azure Kubernetes Service (AKS) enabled by Azure Arc is a managed Kubernetes service that lets you easily deploy workload identity enabled Kubernetes clusters. This article describes how to perform the following tasks:
18+
19+
- Create an AKS Arc cluster with workload identity enabled (preview).
20+
- Create a Kubernetes service account and bind it to the Azure Managed Identity.
21+
- Create a federated credential on the managed identity to trust the OIDC issuer.
22+
- Deploy your application.
23+
- Example: Grant a pod in the cluster access to secrets in an Azure key vault.
24+
25+
26+
> [!IMPORTANT]
27+
> These preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. Azure Kubernetes Service, enabled by Azure Arc previews are partially covered by customer support on a best-effort basis.
28+
29+
> [!NOTE]
30+
> In public preview, AKS on Azure Stack HCI supports enabling workload identity during AKS cluster creation. However, enabling workload identity after cluster creation or disabling it afterward is currently unsupported.
31+
32+
## Prerequisites
33+
34+
Before you deploy a Kubernetes cluster with Azure Arc enabled, you must have the following prerequisites:
35+
36+
- If you don't have an Azure subscription, create an [Azure free account](https://azure.microsoft.com/free/?ref=microsoft.com&utm_source=microsoft.com&utm_medium=docs&utm_campaign=visualstudio) before you begin.
37+
- This article requires version 1.4.23 or later of the Azure CLI. If you're using Azure Cloud Shell, the latest version is already installed.
38+
39+
### Export environment variables
40+
41+
To help simplify the steps to configure the identities required, the following commands define environment variables that are referenced in the examples in this article. Replace the following values with your own:
42+
43+
```azurecli
44+
$AZSubscriptionID = "00000000-0000-0000-0000-000000000000"
45+
$Location = "westeurope"
46+
$resource_group_name = "myResourceGroup"
47+
48+
$aks_cluster_name = "myAKSCluster"
49+
50+
$SERVICE_ACCOUNT_NAMESPACE = "default"
51+
$SERVICE_ACCOUNT_NAME = "workload-identity-sa"
52+
53+
$FedIdCredentialName = "myFedIdentity"
54+
$MSIName = "myIdentity"
55+
56+
# To access key vault secrets from a pod in the cluster, include these variables:
57+
$KVName = "KV-workload-id"
58+
$KVSecretName= "KV-secret"
59+
```
60+
61+
### Set the active subscription
62+
63+
First, set your subscription as the current active subscription. Run the [az account set](/cli/azure/account#az-account-set) command with your subscription ID:
64+
65+
```azurecli
66+
az login
67+
az account set -s $AZSubscriptionID
68+
```
69+
70+
### Create a resource group
71+
72+
An [Azure resource group](/azure/azure-resource-manager/management/overview) is a logical group in which Azure resources are deployed and managed. When you create a resource group, you're prompted to specify a location. This location is the storage location of your resource group metadata and where your resources run in Azure if you don't specify another region during resource creation.
73+
74+
To create a resource group, run the [az group create](/cli/azure/group#az-group-create) command:
75+
76+
```azurecli
77+
az group create --name $resource_group_name --location $Location
78+
```
79+
80+
The following example output shows the successful creation of a resource group:
81+
82+
```json
83+
{
84+
"id": "/subscriptions/<guid>/resourceGroups/myResourceGroup",
85+
"location": "westeurope",
86+
"managedBy": null,
87+
"name": "$resource_group_name",
88+
"properties": {
89+
"provisioningState": "Succeeded"
90+
},
91+
"tags": null
92+
}
93+
```
94+
95+
## Step 1: Create an AKS Arc cluster with workload identity enabled
96+
97+
To create an AKS Arc cluster, you need both the `$customlocation_ID` and `$logicnet_Id` values.
98+
99+
- `$customlocation_ID`: The Azure Resource Manager ID of the custom location. The custom location is configured during the Azure Stack HCI cluster deployment. Your infrastructure admin should give you the Resource Manager ID of the custom location. You can also get the Resource Manager ID using `$customlocation_ID = $(az customlocation show --name "<your-custom-location-name>" --resource-group $resource_group_name --query "id" -o tsv)`, if the infrastructure admin provides a custom location name and resource group name.
100+
- `$logicnet_Id`: The Azure Resource Manager ID of the Azure Local logical network created [following these steps](/azure/aks/hybrid/aks-networks?tabs=azurecli). Your infrastructure admin should give you the Resource Manager ID of the logical network. You can also get the Resource Manager ID using `$logicnet_Id = $(az stack-hci-vm network lnet show --name "<your-lnet-name>" --resource-group $resource_group_name --query "id" -o tsv)`, if the infrastructure admin provides a logical network name and resource group name.
101+
102+
Run the [az aksarc create](/cli/azure/aksarc#az-aksarc-create) command with the `--enable-oidc-issuer --enable-workload-identity` parameter. Provide your **entra-admin-group-object-ids** and ensure you're a member of the Microsoft Entra ID admin group for proxy mode access:
103+
104+
```azurecli
105+
az aksarc create
106+
-n $aks_cluster_name -g $resource_group_name
107+
--custom-location $customlocation_ID --vnet-ids $logicnet_Id
108+
--aad-admin-group-object-ids <entra-admin-group-object-ids>
109+
--generate-ssh-keys
110+
--enable-oidc-issuer --enable-workload-identity
111+
```
112+
113+
After a few minutes, the command completes and returns JSON-formatted information about the cluster.
114+
115+
It might take some time for the workload identity extension to be deployed after successfully creating a provisioned cluster. Use the following command to check the workload identity extension status:
116+
117+
```azurecli
118+
az connectedk8s show -n $aks_cluster_name -g $resource_group_name
119+
```
120+
121+
```json
122+
# agentState = "Succeeded"
123+
"agentPublicKeyCertificate": "",
124+
"agentVersion": "1.21.10",
125+
"arcAgentProfile": {
126+
"agentAutoUpgrade": "Enabled",
127+
"agentErrors": [],
128+
"agentState": "Succeeded",
129+
"desiredAgentVersion": "",
130+
"systemComponents": null
131+
132+
# oidcIssuerProfile "enabled": true and "issuerUrl" present
133+
134+
"oidcIssuerProfile": {
135+
"enabled": true,
136+
"issuerUrl": "https://oidcdiscovery-{location}-endpoint-1111111111111111.000.azurefd.net/00000000-0000-0000-0000-000000000000/11111111-1111-1111-1111-111111111111/"}
137+
```
138+
139+
In the Azure portal, you can view the **wiextension** extension under the **Properties** section of your Kubernetes cluster.
140+
141+
> [!IMPORTANT]
142+
> As part of the security enhancement for AKS Arc clusters, workload identity enablement triggers two changes. First, the Kubernetes service account signing key automatically rotates every 45 days and remains valid for 90 days. Second, the `--service-account-extend-token-expiration` flag is disabled, reducing token validity from one year to a maximum of 24 hours.
143+
144+
### Save the OIDC issuer URL to an environment variable
145+
146+
Once AKS cluster is created successfully, you can get the OIDC issuer URL and save it to an environment variable. Run the following command:
147+
148+
```azurecli
149+
$SERVICE_ACCOUNT_ISSUER =$(az connectedk8s show --n $aks_cluster_name --resource-group $resource_group_name --query "oidcIssuerProfile.issuerUrl" --output tsv)
150+
```
151+
152+
## Step 2: Create a Kubernetes service account and bind it to the Azure Managed Identity
153+
154+
First, create a managed identity. Run the [az identity create](/cli/azure/identity#az-identity-create) command:
155+
156+
```azurecli
157+
az identity create --name $MSIName --resource-group $resource_group_name --location $Location --subscription $AZSubscriptionID
158+
```
159+
160+
Next, create variables for the managed identity's client ID:
161+
162+
```azurecli
163+
$MSIId=$(az identity show --resource-group $resource_group_name --name $MSIName --query 'clientId' --output tsv)
164+
$MSIPrincipalId=$(az identity show --resource-group $resource_group_name --name $MSIName --query 'principalId' --output tsv)
165+
```
166+
167+
### Create a Kubernetes service account
168+
169+
Create a Kubernetes service account and annotate it with the client ID of the managed identity created in the previous step:
170+
171+
```azurecli
172+
az connectedk8s proxy -n $aks_cluster_name -g $resource_group_name
173+
```
174+
175+
Open a new window. Copy and paste the following CLI commands:
176+
177+
```azurecli
178+
$yaml = @" apiVersion: v1 kind: ServiceAccount metadata: annotations: azure.workload.identity/client-id: $MSIId name: $SERVICE_ACCOUNT_NAME namespace: $SERVICE_ACCOUNT_NAMESPACE "@ $yaml = $yaml -replace '\$MSIId', $MSIId ` -replace '\$SERVICE_ACCOUNT_NAME', $SERVICE_ACCOUNT_NAME ` -replace '\$SERVICE_ACCOUNT_NAMESPACE', $SERVICE_ACCOUNT_NAMESPACE $yaml | kubectl apply -f -
179+
```
180+
181+
The following output shows successful creation of the service account:
182+
183+
```output
184+
serviceaccount/workload-identity-sa created
185+
```
186+
187+
## Step 3: Create a federated credential on the managed identity to trust the OIDC issuer
188+
189+
First, create a federated identity credential. Call the [az identity federated-credential create](/cli/azure/identity/federated-credential#az-identity-federated-credential-create) command
190+
to create the federated identity credential between the managed identity, the service account issuer, and the subject. For more information about federated identity credentials in Microsoft Entra,
191+
see [Overview of federated identity credentials in Microsoft Entra ID](/graph/api/resources/federatedidentitycredentials-overview).
192+
193+
```azurecli
194+
# Create a federated credential
195+
196+
az identity federated-credential create --name $FedIdCredentialName --identity-name $MSIName --resource-group $resource_group_name --issuer $SERVICE_ACCOUNT_ISSUER --subject "system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}"
197+
198+
# Show the federated credential
199+
200+
az identity federated-credential show --name $FedIdCredentialName --resource-group $resource_group_name --identity-name $MSIName
201+
```
202+
203+
> [!NOTE]
204+
> After you add a federated identity credential, it takes a few seconds to propagate. Token requests made immediately afterward might fail until the cache refreshes. To prevent this issue, consider adding a brief delay after creating the federated identity credential.
205+
206+
## Step 4: Deploy your application
207+
208+
When you deploy your application pods, the manifest should reference the service account created in the **Create Kubernetes service account** step. The following manifest shows how to reference the account, specifically the `metadata\namespace` and `spec\serviceAccountName` properties. Make sure to specify an image for `image` and a container name
209+
for `containerName`:
210+
211+
```azurecli
212+
$image = "<image>" # Replace <image> with the actual image name
213+
$containerName = "<containerName>" # Replace <containerName> with the actual container name
214+
215+
$yaml = @"
216+
apiVersion: v1
217+
kind: Pod
218+
metadata:
219+
name: sample-quick-start
220+
namespace: $SERVICE_ACCOUNT_NAMESPACE
221+
labels:
222+
azure.workload.identity/use: "true" # Required. Only pods with this label can use workload identity.
223+
spec:
224+
serviceAccountName: $SERVICE_ACCOUNT_NAME
225+
containers:
226+
- image: $image
227+
name: $containerName
228+
"@
229+
230+
# Replace variables within the YAML content
231+
$yaml = $yaml -replace '\$SERVICE_ACCOUNT_NAMESPACE', $SERVICE_ACCOUNT_NAMESPACE `
232+
-replace '\$SERVICE_ACCOUNT_NAME', $SERVICE_ACCOUNT_NAME
233+
234+
# Apply the YAML configuration
235+
$yaml | kubectl apply -f -
236+
```
237+
238+
> [!IMPORTANT]
239+
> Ensure that the application pods using workload identity include the label `azure.workload.identity/use: "true"` in the pod spec. Otherwise the pods fail after they restart.
240+
241+
## Example: Grant permissions to access Azure Key Vault
242+
243+
The instructions in this step describe how to access secrets, keys, or certificates in an Azure key vault from the pod. The examples in this section configure access to secrets in the key vault for the workload identity, but you can perform similar steps to configure access to keys or certificates.
244+
245+
The following example shows how to use the Azure role-based access control (Azure RBAC) permission model to grant the pod access to the key vault. For more information about the Azure RBAC permission model for Azure Key Vault, see [Grant permission to applications to access an Azure key vault using Azure RBAC](/azure/key-vault/general/rbac-guide).
246+
247+
1. Create a key vault with purge protection and RBAC authorization enabled. You can also use an existing key vault if it is configured for both purge protection and RBAC authorization:
248+
249+
```azurecli
250+
az keyvault create --name $KVName --resource-group $resource_group_name --location $Location --enable-purge-protection --enable-rbac-authorization
251+
252+
# retrieve the key vault ID for role assignment
253+
$KVId=$(az keyvault show --resource-group $resource_group_name --name $KVName --query id --output tsv)
254+
```
255+
256+
1. Assign the RBAC [Key Vault Secrets Officer](/azure/role-based-access-control/built-in-roles/security#key-vault-secrets-officer) role to yourself so that you can create a secret in the new key vault. New role assignments can take up to five minutes to propagate and be updated by the authorization server.
257+
258+
```azurecli
259+
az role assignment create --assignee-object-id $MSIPrincipalId --role "Key Vault Secrets Officer" --scope $KVId --assignee-principal-type ServicePrincipal
260+
```
261+
262+
1. Create a secret in the key vault:
263+
264+
```azurecli
265+
az keyvault secret set --vault-name $KVName --name $KVSecretName --value "Hello!"
266+
```
267+
268+
1. Assign the [Key Vault Secrets User](/azure/role-based-access-control/built-in-roles/security#key-vault-secrets-user) role to the user-assigned managed identity that you created previously. This step gives the managed identity permission to read secrets from the key vault:
269+
270+
```azurecli
271+
az role assignment create --assignee-object-id $MSIPrincipalId --role "Key Vault Secrets User" --scope $KVId --assignee-principal-type ServicePrincipal
272+
```
273+
274+
1. Create an environment variable for the key vault URL:
275+
276+
```azurecli
277+
$KVUrl=$(az keyvault show --resource-group $resource_group_name --name $KVName --query properties.vaultUri --output tsv)
278+
```
279+
280+
1. Deploy a pod that references the service account and key vault URL:
281+
282+
```azurecli
283+
$yaml = @"
284+
apiVersion: v1
285+
kind: Pod
286+
metadata:
287+
name: sample-quick-start
288+
namespace: $SERVICE_ACCOUNT_NAMESPACE
289+
labels:
290+
azure.workload.identity/use: "true"
291+
spec:
292+
serviceAccountName: $SERVICE_ACCOUNT_NAME
293+
containers:
294+
- image: ghcr.io/azure/azure-workload-identity/msal-go
295+
name: oidc
296+
env:
297+
- name: KEYVAULT_URL
298+
value: $KVUrl
299+
- name: SECRET_NAME
300+
value: $KVSecretName
301+
nodeSelector:
302+
kubernetes.io/os: linux
303+
"@
304+
305+
# Replace variables within the YAML content
306+
$yaml = $yaml -replace '\$SERVICE_ACCOUNT_NAMESPACE', $SERVICE_ACCOUNT_NAMESPACE `
307+
-replace '\$SERVICE_ACCOUNT_NAME', $SERVICE_ACCOUNT_NAME `
308+
-replace '\$KVUrl', $KVUrl `
309+
-replace '\$KVSecretName', $KVSecretName
310+
311+
# Apply the YAML configuration
312+
$yaml | kubectl --kubeconfig $aks_cluster_name apply -f -
313+
```
314+
315+
## Next steps
316+
317+
In this article, you deployed a Kubernetes cluster and configured it to use a workload identity in preparation for application workloads to
318+
authenticate with that credential. Now you're ready to deploy your application and configure it to use the workload identity with the latest version of the [Azure Identity client library](/azure/active-directory/develop/reference-v2-libraries).

0 commit comments

Comments
 (0)