Skip to content

Commit 68ef3e8

Browse files
Stacyrch140Stacyrch140
authored
Merge pull request #268454 from qpetraroia/main
Deployment safeguards public preview docs
2 parents 8def4bd + 601af17 commit 68ef3e8

File tree

2 files changed

+203
-0
lines changed

2 files changed

+203
-0
lines changed

‎articles/aks/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -214,6 +214,8 @@
214214
href: ../backup/azure-kubernetes-service-backup-overview.md?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json
215215
- name: Windows containers
216216
href: windows-best-practices.md
217+
- name: Enforce best practices with deployment safeguards
218+
href: deployment-safeguards.md
217219
- name: How-to guides
218220
items:
219221
- name: Migration

‎articles/aks/deployment-safeguards.md

Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
---
2+
title: Use deployment safeguards to enforce best practices
3+
description: Learn how to use deployment safeguards to enforce best practices on an Azure Kubernetes Service (AKS) cluster.
4+
author: nickomang
5+
ms.topic: how-to
6+
ms.date: 03/11/2024
7+
ms.author: nickoman
8+
---
9+
10+
# Deployment safeguards (preview)
11+
12+
Throughout the development lifecycle, it's common for bugs, issues, and other problems to arise if the initial deployment of your Kubernetes resources includes misconfigurations. To ease the burden of Kubernetes development, Azure Kubernetes Service (AKS) offers deployment safeguards (preview). Deployment safeguards enforce Kubernetes best practices in your AKS cluster through Azure Policy controls.
13+
14+
Deployment safeguards offer two levels of configuration. The `Warning` level populates warning messages in the code terminal when a cluster isn't following best practices. It lets you know that your cluster configuration is noncompliant, but allows the request to go through. The `Enforcement` level enforces compliant configurations, denying deployments if they aren't following best practices.
15+
16+
After you configure deployment safeguards for 'Warning' or 'Enforcement', Deployment safeguards programmatically assess your clusters at creation or update time for compliance. Deployment safeguards also display aggregated compliance information across your workloads at a per resource level via Azure Policy's compliance dashboard in the [Azure portal][Azure-Policy-compliance-portal] or in your CLI or terminal. Running a noncompliant workload indicates that your cluster isn't following best practices and that workloads on your cluster are at risk of experiencing issues caused by your cluster configuration.
17+
18+
[!INCLUDE [preview features callout](./includes/preview/preview-callout.md)]
19+
20+
## Before you get started
21+
22+
Before you configure deployment safeguards, make sure that your environment meets the following requirements.
23+
24+
### Prerequisites
25+
26+
- Azure Policy's add-on for AKS must be enabled. For more information, see [Enabling Azure Policy on your AKS cluster][policy-for-kubernetes].
27+
28+
- To configure deployment safeguards, you must have version `2.0.0b1` or later of the `aks-preview` extension. We recommend that you install the latest version of Azure CLI as well as the `aks-preview` CLI extension.
29+
30+
- In order to create and modify the configuration for deployment safeguards, you need a subscription with the [following permissions on your AKS cluster][Azure-Policy-RBAC-permissions]:
31+
32+
- *Microsoft.Authorization/policyAssignments/write*
33+
34+
- *Microsoft.Authorization/policyAssignments/read*
35+
36+
#### Install the aks-preview CLI extension
37+
38+
1. Install the `aks-preview` CLI extension using the [az extension add][az-extension-add] command.
39+
40+
```azurecli-interactive
41+
az extension add --name aks-preview
42+
```
43+
44+
2. Update the extension to ensure you have the latest version installed using the [`az extension update`][az-extension-update] command.
45+
46+
```azurecli-interactive
47+
az extension update --name aks-preview
48+
```
49+
50+
#### Register the feature flag for deployment safeguards
51+
52+
Register the `SafeguardsPreview` feature flag by using the [az feature register][az-feature-register] command, as shown in the following example:
53+
54+
```azurecli-interactive
55+
az feature register --namespace Microsoft.ContainerService --name SafeguardsPreview
56+
```
57+
58+
It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [az feature show][az-feature-show] command:
59+
60+
```azurecli-interactive
61+
az feature show --namespace Microsoft.ContainerService --name SafeguardsPreview
62+
```
63+
64+
When the status reflects *Registered*, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register][az-provider-register] command:
65+
66+
```azurecli-interactive
67+
az provider register --namespace Microsoft.ContainerService
68+
```
69+
70+
## Deployment safeguards policies
71+
72+
The following table lists the policies that become active when you enable deployment safeguards and the Kubernetes resource that they will target. You can see the [currently available deployment safeguards][deployment-safeguards-list] in the Azure portal as an Azure Policy definition, or view them at [Azure Policy built-in definitions for Azure Kubernetes Service][Azure-Policy-built-in-definition-docs]. The intention behind this collection is to create a common and generic list of best practices applicable to most users and use cases.
73+
74+
| Deployment safeguard policies | Kubernetes resource that is targeted |
75+
|--------------|--------------|
76+
| [Preview]: Cannot Edit Individual Nodes | Node |
77+
| Kubernetes cluster containers CPU and memory resource limits shouldn't exceed the specified limits | Pod |
78+
| [Preview]: Must Have Anti Affinity Rules Set | Deployment, StatefulSet, ReplicationController, ReplicaSet |
79+
| [Preview]: No AKS Specific Labels | Deployment, StatefulSet, Replicaset |
80+
| Kubernetes cluster containers should only use allowed images | Pod |
81+
| [Preview]: Reserved System Pool Taints | Node |
82+
| Ensure cluster containers have readiness or liveness probes configured | Pod |
83+
| Kubernetes clusters should use Container Storage Interface(CSI) driver StorageClass | StorageClass |
84+
| [Preview]: Kubernetes cluster containers should only pull images when image pull secrets are present | Pod |
85+
| [Preview]: Kubernetes cluster should implement accurate Pod Disruption Budgets | Deployment, ReplicaSet, StatefulSet |
86+
| [Preview]: Kubernetes cluster services should use unique selectors | Service |
87+
88+
If you would like to submit an idea or request for deployment safeguards, open an issue in the [AKS GitHub repository][aks-gh-repo] and add `[deployment safeguards request]` to the beginning of the title.
89+
90+
## Enable deployment safeguards
91+
92+
>[!NOTE]
93+
> If you have enabled Azure Policy for the first time to use deployment safeguards, you may need to wait up to 35 minutes for Azure Policy to take effect.
94+
>
95+
> By using deployment safeguards in `Enforcement` mode, you are opting in to your deployments being blocked as well. Please be aware of how these policies will work with your AKS cluster before you enable `Enforcement`.
96+
97+
To enable deployment safeguards on a new cluster, include the `--safeguards-level` flag when you create the cluster.
98+
99+
To receive warnings, set the `--safeguards-level` to "Warning". To deny all deployments that don't adhere to deployment safeguards, set the `--safeguards-level` to "Enforcement".
100+
101+
```azurecli-interactive
102+
az aks create --name myAKSCluster --resource-group myResourceGroup --enable-addons azure-policy --safeguards-level Warning
103+
```
104+
105+
You can also update an existing cluster to enable deployment safeguards, assuming that you have already enabled the Azure Policy add-on for the cluster.
106+
107+
```azurecli-interactive
108+
az aks update --name myAKSCluster --resource-group myResourceGroup --safeguards-level Enforcement
109+
```
110+
111+
### Excluding namespaces
112+
113+
You can also exclude certain namespaces from deployment safeguards. When a namespace is excluded, activity in that namespace is unaffected by deployment safeguards warnings or enforcement.
114+
115+
For example, to exclude the namespaces `ns1` and `ns2`, use a comma-separated list.
116+
117+
```azurecli-interactive
118+
az aks update --name myAKSCluster --resource-group myResourceGroup --safeguards-level Warning --safeguards-excluded-ns ns1,ns2
119+
```
120+
121+
## Verify compliance across clusters via your CLI or terminal
122+
123+
After deploying your Kubernetes manifest, if the cluster isn't compliant with deployment safeguards, then you'll see warnings or a potential denial message in your CLI or terminal. The following examples show what that experience might look like for you.
124+
125+
**Warning**
126+
127+
```
128+
PS C:\Users\testUser\Code> kubectl apply -f pod.yml
129+
Warning: [azurepolicy-k8sazurev2containerenforceprob-0e8a839bcd103e7b96a8] Container <my-container> in your Pod <my-pod> has no <livenessProbe>. Required probes: ["readinessProbe", "livenessProbe"]
130+
Warning: [azurepolicy-k8sazurev2containerenforceprob-0e8a839bcd103e7b96a8] Container <my-container> in your Pod <my-pod> has no <readinessProbe>. Required probes: ["readinessProbe", "livenessProbe"]
131+
Warning: [azurepolicy-k8sazurev1restrictedlabels-67c4210cc58f28acdfdb] Label <{"kubernetes.azure.com"}> is reserved for AKS use only
132+
Warning: [azurepolicy-k8sazurev3containerlimits-a8754961dbd4c1d8b49d] container <my-container> has no resource limits
133+
Warning: [azurepolicy-k8sazurev1containerrestrictedi-bde07e1776cbcc9aa8b8] my-pod in default does not have imagePullSecrets. Unauthenticated image pulls are not recommended.
134+
pod/my-pod created
135+
```
136+
137+
**Enforcement**
138+
139+
```
140+
PS C:\Users\testUser\Code> kubectl apply -f pod.yml
141+
Error from server (Forbidden): error when creating ".\pod.yml": admission webhook "validation.gatekeeper.sh" denied the request: [azurepolicy-k8sazurev2containerenforceprob-0e8a839bcd103e7b96a8] Container <my-container> in your Pod <my-pod> has no <livenessProbe>. Required probes: ["readinessProbe", "livenessProbe"]
142+
[azurepolicy-k8sazurev2containerenforceprob-0e8a839bcd103e7b96a8] Container <my-container> in your Pod <my-pod> has no <readinessProbe>. Required probes: ["readinessProbe", "livenessProbe"]
143+
[azurepolicy-k8sazurev2containerallowedimag-1ff6d14b2f8da22019d7] Container image my-image for container my-container has not been allowed.
144+
[azurepolicy-k8sazurev1restrictedlabels-67c4210cc58f28acdfdb] Label <{"kubernetes.azure.com"}> is reserved for AKS use only
145+
[azurepolicy-k8sazurev3containerlimits-a8754961dbd4c1d8b49d] container <my-container> has no resource limits
146+
[azurepolicy-k8sazurev1containerrestrictedi-bde07e1776cbcc9aa8b8] my-pod in default does not have imagePullSecrets. Unauthenticated image pulls are not recommended.
147+
```
148+
149+
## Verify compliance across clusters via the Azure Policy dashboard
150+
151+
To verify deployment safeguards have been applied and to check on your cluster's compliance, navigate to the Azure portal page for your cluster and select **Policies**, then select **go to Azure Policy**.
152+
153+
From the list of policies and initiatives, select the initiative associated with deployment safeguards. You'll see a dashboard showing compliance state across your AKS cluster.
154+
155+
> [!NOTE]
156+
> To properly assess compliance across your AKS cluster, the Azure Policy initiative must be scoped to your cluster's resource group.
157+
158+
## Disable deployment safeguards
159+
160+
To disable deployment safeguards on your cluster, set the `--safeguards-level` to `Off`.
161+
162+
```azurecli-interactive
163+
az aks update --name myAKSCluster --resource-group myResourceGroup --safeguards-level Off
164+
```
165+
166+
--
167+
168+
## FAQ
169+
170+
#### I enabled deployment safeguards with Azure Policy for the first time. Why don't I see any warnings? Why aren't my pods being declined?
171+
172+
Azure Policy can take up to 35 minutes to sync with your cluster after it is enabled for the first time.
173+
174+
#### I just switched from Warning to Enforcement. Will this take effect immediately?
175+
176+
When switching deployment safeguard levels, you may need to wait up to 15 minutes for the new level to take effect.
177+
178+
#### Why did my deployment resource get admitted even though it wasn't following best practices?
179+
180+
Deployment safeguards enforce best practice standards through Azure Policy controls and has policies that validate against Kubernetes resources. To evaluate and enforce cluster components, Azure Policy extends [Gatekeeper](https://open-policy-agent.github.io/gatekeeper/website/). Gatekeeper enforcement also currently operates in a [`fail-open` model](https://open-policy-agent.github.io/gatekeeper/website/docs/failing-closed/#considerations). As there's no guarantee that Gatekeeper will respond to our networking call, we make sure that in that case, the validation is skipped so that the deny doesn't block your deployments.
181+
182+
To learn more, see [workload validation in Gatekeeper](https://open-policy-agent.github.io/gatekeeper/website/docs/workload-resources/).
183+
184+
## Next steps
185+
186+
- Learn more about [best practices][best-practices] for operating an AKS cluster.
187+
188+
<!-- LINKS -->
189+
190+
[az-extension-add]: /cli/azure/extension#az-extension-add
191+
[az-extension-update]: /cli/azure/extension#az-extension-update
192+
[best-practices]: ./best-practices.md
193+
[az-provider-register]: /cli/azure/provider#az-provider-register
194+
[az-feature-register]: /cli/azure/feature#az-feature-register
195+
[az-feature-show]: /cli/azure/feature#az-feature-show
196+
[aks-gh-repo]: https://github.com/Azure/AKS
197+
[policy-for-kubernetes]: /azure/governance/policy/concepts/policy-for-kubernetes#install-azure-policy-add-on-for-aks
198+
[deployment-safeguards-list]: https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/InitiativeDetail.ReactView/id/%2Fproviders%2FMicrosoft.Authorization%2FpolicySetDefinitions%2Fc047ea8e-9c78-49b2-958b-37e56d291a44/scopes/
199+
[Azure-Policy-built-in-definition-docs]: /azure/aks/policy-reference#policy-definitions
200+
[Azure-Policy-compliance-portal]: https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/PolicyMenuBlade/~/Compliance
201+
[Azure-Policy-RBAC-permissions]: /azure/governance/policy/overview#azure-rbac-permissions-in-azure-policy

0 commit comments

Comments
 (0)