-
Notifications
You must be signed in to change notification settings - Fork 358
Identity bindings preview doc #5217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
0ecc96b
cd62eea
814ecbf
db8d882
4134e67
22a6ebd
89c633a
0eb5988
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| # Identity Bindings Concepts | ||
|
|
||
| ## Context | ||
|
|
||
| [Workload identity feature exists for AKS today](https://learn.microsoft.com/azure/aks/workload-identity-overview) but has scale limitations in not being able to go beyond 20 Federated Identity Credentials (FICs) per identity. For customers having large scale K8s platform spanning across more than 20 clusters (thus more than 20 issuers => more than 20 FICs required) or having a lot of `<namespace, service-account>` combinations to create FIC mappings for the same identity, this a blocker today. | ||
|
|
||
| Identity Binding is an evolution of workload identity on AKS to provide a more scalable and secure experience. For users requiring Entra authentication from workloads running on AKS, identity binding is recommended as it presents a scalable solution as users scale their application across multiple Kubernetes clusters. | ||
|
|
||
| ## Conceptual Introduction | ||
|
|
||
| An Identity binding is a mapping resource that allows customers to create mapping between one user-assigned managed identity (UAMI) and one AKS cluster where there are workloads that require authentication to Entra using this user assigned managed identity. So let's say you have a UAMI MI-1 which needs to be used in AKS cluster AKS-1, AKS-2, AKS-3,.... you'll create the following identity binding mappings: | ||
|
|
||
| - IB-A mapping MI-1 to AKS-1 | ||
| - IB-B mapping MI-1 to AKS-2 | ||
| - IB-C mapping MI-1 to AKS-3 | ||
|
|
||
| Even if the same user-assigned managed identity is needed by workloads running across multiple clusters, only one Federated Identity Credential is created for the user-assigned managed identity, thus addressing the 20 FIC limitation. When the cluster operator creates an identity binding, this single federated identity credential is automatically created by AKS for this user assigned managed identity. | ||
benjaminapetersen marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
|  | ||
|
|
||
| While the above step allows for usage of a particular user-assigned managed identity within the cluster, the cluster operator is then expected to create ClusterRole/ClusterRoleBinding identifying the namespace and the service accounts (either granularly or collectively across namespace or cluster) that are allowed to use this user-assigned managed identity for Entra authentication. | ||
|
|
||
| ## FAQ | ||
|
|
||
| ### Is identity sameness (namespace and service account sameness) required across clusters when the same user-assigned managed identity is used across clusters? | ||
|
|
||
| No. Identity binding doesn't have any pre-requisite for identity sameness and it's completely up to the cluster operator to explicitly authorize the namespaces/service-accounts inside the cluster that are allowed to use a managed identity. | ||
|
|
||
| ### Can multiple identity bindings be created for the same user-assigned managed identity? | ||
|
|
||
| Yes. The OIDC issuer URL maintained by AKS will be the same for this user-assigned managed identity across all of these identity bindings. | ||
|
|
||
| ### What permissions are required for creation of the identity bindings? | ||
|
|
||
| **ARM permissions required:** | ||
| - `Microsoft.ContainerService/managedClusters/identityBindings/*` | ||
| - `Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/*` (when identityBindings resource is created, federatedIdentityCredentials is automatically created on-behalf-of the cluster operator. if cluster operator who is creating identityBindings doesn't have permissions to create federatedIdentityCredentials, the identityBindings creation will fail) | ||
|
|
||
| **K8s permissions required:** | ||
| - permissions to create ClusterRole and ClusterRoleBinding (which would be possible for cluster admin) | ||
|
|
||
| ### With federated identity credentials auto generated at the time of identity binding creation, when all identity bindings mapping to a MI are deleted, what happens to the FIC that was created for that specific MI? | ||
|
|
||
| There's currently no garbage collection for the FIC when the last identity binding mapped to a MI is deleted. Cluster operator should clean up the FIC only after verifying that all identity-bindings mapped to a MI are deleted, failing which any identity-binding and cluster still dependent on the same would be impacted. | ||
|
|
||
| ### What are networking pre-requisites for usage of identity bindings feature? | ||
|
|
||
| Before identity bindings, workload identity feature required egress to be allowed for `login.microsoftonline.com` so that workloads could send requests to eSTS to exchange service account tokens for Entra access tokens. | ||
|
|
||
| With the identity binding feature, there's no need for `login.microsoftonline.com` as token exchange requests are routed via a cluster-specific identity binding specific proxy deployed on AKS service. | ||
|
|
||
| ## Next Steps | ||
|
|
||
| [Set up identity bindings](./identity-bindings-how-to-guide.md) based on the how-to guide. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,188 @@ | ||
|
|
||
| # Identity Bindings - Getting started | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| * Fill out this [form](https://aka.ms/aks/identity-bindings/private-preview-form) to provide subscription(s) where AKS needs to enable the feature flag (Microsoft.ContainerService/IdentityBinding) for this private preview. | ||
|
|
||
| * Install `aks-preview` Azure CLI extension of version >= `18.0.0b26`: | ||
| ```bash | ||
| # Install the aks-preview extension | ||
| az extension add --name aks-preview | ||
|
|
||
| # Update to the latest version if already installed | ||
| az extension update --name aks-preview | ||
| ``` | ||
|
|
||
| * To complete the set up and usage of identity bindings, the following permissions are required: | ||
| * Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/write | ||
| * Microsoft.ContainerService/managedClusters/write | ||
|
|
||
| ## Setup - AKS Control Plane Resource | ||
|
|
||
| In this section you will map a managed identity to an AKS cluster using an identity binding: | ||
|
|
||
| 1. Create necessary test resources (resource group, AKS cluster, managed identity): | ||
|
|
||
| ```bash | ||
| # Set up environment variables | ||
| export RESOURCE_GROUP="ib-test" | ||
| export LOCATION="westus2" | ||
| export CLUSTER="ibtest" | ||
| export MI_NAME="ib-test-mi" | ||
|
|
||
| # Create resource group, AKS cluster, and identity bindings | ||
| az group create --name $RESOURCE_GROUP -l $LOCATION | ||
| az aks create -g $RESOURCE_GROUP -n $CLUSTER -l $LOCATION --no-ssh-key --enable-workload-identity | ||
| az identity create -g $RESOURCE_GROUP -n $MI_NAME | ||
| ``` | ||
|
|
||
| 2. Create identity binding to map a managed identity to an AKS cluster. | ||
|
|
||
| ```bash | ||
| export MI_RESOURCE_ID=$(az identity show -g $RESOURCE_GROUP -n $MI_NAME --query id -o tsv) | ||
| az aks identity-binding create -g $RESOURCE_GROUP --cluster-name $CLUSTER -n "$MI_NAME-ib" --managed-identity-resource-id $MI_RESOURCE_ID | ||
| ``` | ||
|
|
||
| 3. AKS creates a unique OIDC issuer per managed identity and populates that in the identity binding resource. Check the identity binding resource to obtain the OIDC issuer URL for this managed identity: | ||
|
|
||
| ```bash | ||
| az aks identity-binding show -g $RESOURCE_GROUP --cluster-name $CLUSTER -n "$MI_NAME-ib" | ||
| ``` | ||
|
|
||
| Output should be similar to: | ||
|
|
||
| ``` | ||
| { | ||
| // omitted other fields | ||
| "oidcIssuer": { | ||
| "oidcIssuerUrl": "https://ib.oic.prod-aks.azure.com/<MI-tenant-id>/<MI-client-id>" | ||
| }, | ||
| // omitted other fields | ||
| } | ||
| ``` | ||
|
|
||
| You should be able to find the FIC with name "aks-identity-binding" created under the managed identity, similar to this: | ||
|
|
||
|  | ||
|
|
||
| Once you see output similar to the above, it confirms that the control plane resources have been successfully created. | ||
|
|
||
| **Note:** This FIC is managed by AKS as part of the identity binding. User should not make changes to this FIC to ensure identity bindings and identity federation are unimpacted. | ||
|
|
||
| ## Setup - In-cluster Resource | ||
|
|
||
| In this section you will create a cluster level role and role binding to grant your application (service account) to make use of the identity binding for acquiring access token from Entra. You will need to have cluster admin or similar permission to create cluster role / cluster role binding resources. | ||
|
|
||
| 1. Create ClusterRole and ClusterRoleBinding to authorize the specific namespaces and service accounts that are allowed to use the managed identities (for which identity binding was created earlier): | ||
|
|
||
| ```bash | ||
| # Obtain credentials needed to access the cluster | ||
| az aks get-credentials -g $RESOURCE_GROUP -n $CLUSTER -a -f "${CLUSTER}.kubeconfig" | ||
| export KUBECONFIG="$(pwd)/${CLUSTER}.kubeconfig" | ||
| export MI_CLIENT_ID=$(az identity show -g $RESOURCE_GROUP -n $MI_NAME --query clientId -o tsv) | ||
|
|
||
| kubectl apply -f - <<EOF | ||
| apiVersion: v1 | ||
| kind: Namespace | ||
| metadata: | ||
| name: demo | ||
| --- | ||
| apiVersion: v1 | ||
| kind: ServiceAccount | ||
| metadata: | ||
| name: demo | ||
| namespace: demo | ||
| --- | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRole | ||
| metadata: | ||
| name: test | ||
| rules: | ||
| - verbs: ["use-managed-identity"] | ||
| apiGroups: ["cid.wi.aks.azure.com"] | ||
| resources: ["$MI_CLIENT_ID"] | ||
| --- | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRoleBinding | ||
| metadata: | ||
| name: test | ||
| roleRef: | ||
| apiGroup: rbac.authorization.k8s.io | ||
| kind: ClusterRole | ||
| name: test | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: demo | ||
| namespace: demo | ||
| EOF | ||
| ``` | ||
|
|
||
| 2. Create service account with required annotations with managed identity information: | ||
|
|
||
| ```bash | ||
| kubectl annotate sa demo -n demo azure.workload.identity/tenant-id=$MI_TENANT_ID | ||
| kubectl annotate sa demo -n demo azure.workload.identity/client-id=$MI_CLIENT_ID | ||
| ``` | ||
|
|
||
| 3. Deploy workload with required labels and annotations: | ||
|
|
||
| ```bash | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: v1 | ||
| kind: Pod | ||
| metadata: | ||
| name: test-shell | ||
| namespace: demo | ||
| labels: | ||
| azure.workload.identity/use: "true" | ||
| annotations: | ||
| azure.workload.identity/use-identity-binding: "true" | ||
| spec: | ||
| serviceAccount: demo | ||
| containers: | ||
| - name: azure-cli | ||
| image: mcr.microsoft.com/azure-cli:cbl-mariner2.0 | ||
| command: ["bash", "-c", "sleep infinity"] | ||
| restartPolicy: Never | ||
| EOF | ||
| ``` | ||
|
|
||
| Make sure the pod namespace, service account name, labels, and annotations are using the correct values. Wait for the pod to be ready: | ||
|
|
||
| ```bash | ||
| kubectl get pod | ||
| ``` | ||
|
|
||
| Expected output should be similar to: | ||
|
|
||
| ``` | ||
| NAME READY STATUS RESTARTS AGE | ||
| test-shell 1/1 Running 0 99s | ||
| ``` | ||
|
|
||
| We can describe the pod to confirm it's being properly configured: | ||
|
|
||
| ```bash | ||
| kubectl describe pod test-shell | ||
| ``` | ||
|
|
||
| If you see the environment variables and mounts similar to the below, then it means the pod is configured correctly. | ||
|
|
||
|  | ||
|
|
||
| 4. Exec into the pod for shell access. Inside the pod shell, use `curl` to obtain Entra access token for the managed identity: | ||
|
|
||
| ```bash | ||
| kubectl exec -it test-shell -- bash | ||
|
|
||
| curl "https://${AZURE_KUBERNETES_SNI_NAME}" --cacert $AZURE_KUBERNETES_CA_FILE --resolve "${AZURE_KUBERNETES_SNI_NAME}:443:10.0.0.1" -d "grant_type=client_credentials" -d "client_assertion_type=urn:ietf:params:oauth:client-assertion-type:jwt-bearer" -d "scope=https://management.azure.com//.default" -d "client_assertion=$(cat $AZURE_FEDERATED_TOKEN_FILE)" -d "client_id=$AZURE_CLIENT_ID" | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @aramase / @shashankbarsin maybe we can publish this https://github.com/bahe-msft/azure-workload-identity/tree/ib-examples/examples/identitybinding-msal-go to
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The code for webhook changes is in a feature branch: https://github.com/Azure/azure-workload-identity/tree/feature/custom-token-endpoint. It would be fine to have the example changes merged to that branch. I don't think we need to build the image for the demo app yet. |
||
| ``` | ||
|
|
||
| You can also paste this token (value of access_token field in the response) in https://jwt.ms/ to get decoded output similar to the following: | ||
|
|
||
|  | ||
|
|
||
| ## Next Steps | ||
|
|
||
| Identity binding allows you to map multiple AKS clusters to the same MI and to go beyond the [20 FIC limit that existed with workload identity federation with Entra](https://learn.microsoft.com/entra/workload-id/workload-identity-federation-considerations#general-federated-identity-credential-considerations). So try creating more identity bindings for more clusters to the same identity. And you can also try allowing more subjects to use the MI within those clusters. | ||
Uh oh!
There was an error while loading. Please reload this page.