|
| 1 | +--- |
| 2 | +title: Use ImageCleaner on Azure Kubernetes Service (AKS) |
| 3 | +description: Learn how to use ImageCleaner to clean up stale images on Azure Kubernetes Service (AKS) |
| 4 | +ms.author: nickoman |
| 5 | +author: nickomang |
| 6 | +services: container-service |
| 7 | +ms.topic: article |
| 8 | +ms.date: 08/26/2022 |
| 9 | +--- |
| 10 | + |
| 11 | +# Use ImageCleaner to clean up stale images on your Azure Kubernetes Service cluster (preview) |
| 12 | + |
| 13 | +It's common to use pipelines to build and deploy images on Azure Kubernetes Service (AKS) clusters. While great for image creation, this process often doesn't account for the stale images left behind and can lead to image bloat on cluster nodes. These images can present security issues as they may contain vulnerabilities. By cleaning these unreferenced images, you can remove an area of risk in your clusters. When done manually, this process can be time intensive, which ImageCleaner can mitigate via automatic image identification and removal. |
| 14 | + |
| 15 | +[!INCLUDE [preview features callout](./includes/preview/preview-callout.md)] |
| 16 | + |
| 17 | +## Prerequisites |
| 18 | + |
| 19 | +* An Azure subscription. If you don't have an Azure subscription, you can create a [free account](https://azure.microsoft.com/free). |
| 20 | +* [Azure CLI][azure-cli-install] or [Azure PowerShell][azure-powershell-install] and the `aks-preview` CLI extension installed. |
| 21 | +* The `EnableImageCleanerPreview` feature flag registered on your subscription: |
| 22 | + |
| 23 | +### [Azure CLI](#tab/azure-cli) |
| 24 | + |
| 25 | +Register the `EnableImageCleanerPreview` feature flag by using the [az feature register][az-feature-register] command, as shown in the following example: |
| 26 | + |
| 27 | +```azurecli-interactive |
| 28 | +az feature register --namespace "Microsoft.ContainerService" --name "EnableImageCleanerPreview" |
| 29 | +``` |
| 30 | + |
| 31 | +It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [az feature list][az-feature-list] command: |
| 32 | + |
| 33 | +```azurecli-interactive |
| 34 | +az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/EnableImageCleanerPreview')].{Name:name,State:properties.state}" |
| 35 | +``` |
| 36 | + |
| 37 | +When ready, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register][az-provider-register] command: |
| 38 | + |
| 39 | +```azurecli-interactive |
| 40 | +az provider register --namespace Microsoft.ContainerService |
| 41 | +``` |
| 42 | + |
| 43 | +### [Azure PowerShell](#tab/azure-powershell) |
| 44 | + |
| 45 | +Register the `EnableImageCleanerPreview` feature flag by using the [Register-AzProviderPreviewFeature][register-azproviderpreviewfeature] cmdlet, as shown in the following example: |
| 46 | + |
| 47 | +```azurepowershell-interactive |
| 48 | +Register-AzProviderPreviewFeature -ProviderNamespace Microsoft.ContainerService -Name EnableImageCleanerPreview |
| 49 | +``` |
| 50 | + |
| 51 | +It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [Get-AzProviderPreviewFeature][get-azproviderpreviewfeature] cmdlet: |
| 52 | + |
| 53 | +```azurepowershell-interactive |
| 54 | +Get-AzProviderPreviewFeature -ProviderNamespace Microsoft.ContainerService -Name EnableImageCleanerPreview | |
| 55 | + Format-Table -Property Name, @{name='State'; expression={$_.Properties.State}} |
| 56 | +``` |
| 57 | + |
| 58 | +When ready, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [Register-AzResourceProvider][register-azresourceprovider] command: |
| 59 | + |
| 60 | +```azurepowershell-interactive |
| 61 | +Register-AzResourceProvider -ProviderNamespace Microsoft.ContainerService |
| 62 | +``` |
| 63 | + |
| 64 | +--- |
| 65 | + |
| 66 | +## Limitations |
| 67 | + |
| 68 | +ImageCleaner does not support the following: |
| 69 | + |
| 70 | +* ARM64 node pools. For more information, see [Azure Virtual Machines with ARM-based processors][arm-vms]. |
| 71 | +* Windows node pools. |
| 72 | + |
| 73 | +## How ImageCleaner works |
| 74 | + |
| 75 | +When enabled, an `eraser-controller-manager` pod is deployed on each agent node, which will use an `ImageList` CRD to determine unreferenced and vulnerable images. Vulnerability is determined based on a [trivy][trivy] scan, after which images with a `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL` classification are flagged. An updated `ImageList` will be automatically generated by ImageCleaner based on a set time interval, and can also be supplied manually. |
| 76 | + |
| 77 | +Once an `ImageList` is generated, ImageCleaner will remove all the images in the list from node VMs. |
| 78 | + |
| 79 | + |
| 80 | +:::image type="content" source="./media/image-cleaner/image-cleaner.jpg" alt-text="A diagram showing ImageCleaner's workflow. The ImageCleaner pods running on the cluster can generate an ImageList, or manual input can be provided."::: |
| 81 | + |
| 82 | +## Configuration options |
| 83 | + |
| 84 | +In addition to choosing between manual and automatic mode, there are several options for ImageCleaner: |
| 85 | + |
| 86 | +|Name|Description|Required| |
| 87 | +|----|-----------|--------| |
| 88 | +|--enable-image-cleaner|Enable the ImageCleaner feature for an AKS cluster|Yes, unless disable is specified| |
| 89 | +|--disable-image-cleaner|Disable the ImageCleaner feature for an AKS cluster|Yes, unless enable is specified| |
| 90 | +|--image-cleaner-interval-hours|This parameter determines the interval time (in hours) ImageCleaner will use to run. The default value is one week, the minimum value is 24 hours and the maximum is three months.|No| |
| 91 | + |
| 92 | +## Enable ImageCleaner on your AKS cluster |
| 93 | + |
| 94 | +To create a new AKS cluster using the default interval, use [az aks create][az-aks-create]: |
| 95 | + |
| 96 | +```azurecli-interactive |
| 97 | +az aks create -g MyResourceGroup -n MyManagedCluster \ |
| 98 | + --enable-image-cleaner |
| 99 | +``` |
| 100 | + |
| 101 | +To enable on an existing AKS cluster, use [az aks update][az-aks-update]: |
| 102 | + |
| 103 | +```azurecli-interactive |
| 104 | +az aks update -g MyResourceGroup -n MyManagedCluster \ |
| 105 | + --enable-image-cleaner |
| 106 | +``` |
| 107 | + |
| 108 | +The `--image-cleaner-interval-hours` parameter can be specified at creation time or for an existing cluster. For example, the following command updates the interval for a cluster with ImageCleaner already enabled: |
| 109 | + |
| 110 | +```azurecli-interactive |
| 111 | +az aks update -g MyResourceGroup -n MyManagedCluster \ |
| 112 | + --image-cleaner-interval-hours 48 |
| 113 | +``` |
| 114 | + |
| 115 | +Based on your configuration, ImageCleaner will generate an `ImageList` containing non-running and vulnerable images at the desired interval. ImageCleaner will automatically remove these images from cluster nodes. |
| 116 | + |
| 117 | +## Manually remove images |
| 118 | + |
| 119 | +To manually remove images from your cluster using ImageCleaner, first create an `ImageList`. For example, save the following as `image-list.yml`: |
| 120 | + |
| 121 | +```yml |
| 122 | +apiVersion: eraser.sh/v1alpha1 |
| 123 | +kind: ImageList |
| 124 | +metadata: |
| 125 | + name: imagelist |
| 126 | +spec: |
| 127 | + images: |
| 128 | + - docker.io/library/alpine:3.7.3 # You can also use "*" to specify all non-running images |
| 129 | +``` |
| 130 | +
|
| 131 | +And apply it to the cluster: |
| 132 | +
|
| 133 | +```bash |
| 134 | +kubectl apply -f image-list.yml |
| 135 | +``` |
| 136 | + |
| 137 | +A job will trigger which causes ImageCleaner to remove the desired images from all nodes. |
| 138 | + |
| 139 | +## Disable ImageCleaner |
| 140 | + |
| 141 | +To stop using ImageCleaner, you can disable it via the `--disable-image-cleaner` flag: |
| 142 | + |
| 143 | +```azurecli-interactive |
| 144 | +az aks update -g MyResourceGroup -n MyManagedCluster |
| 145 | + --disable-image-cleaner |
| 146 | +``` |
| 147 | + |
| 148 | +## Logging |
| 149 | + |
| 150 | +The deletion logs are stored in the `image-cleaner-kind-worker` pods. You can check these via `kubectl logs` or via the Container Insights pod log table if the [Azure Monitor add-on](./monitor-aks.md) is enabled. |
| 151 | + |
| 152 | +<!-- LINKS --> |
| 153 | + |
| 154 | +[azure-cli-install]: /cli/azure/install-azure-cli |
| 155 | +[azure-powershell-install]: /powershell/azure/install-az-ps |
| 156 | + |
| 157 | +[az-aks-create]: /cli/azure/aks#az_aks_create |
| 158 | +[az-aks-update]: /cli/azure/aks#az_aks_update |
| 159 | +[az-feature-register]: /cli/azure/feature#az_feature_register |
| 160 | +[register-azproviderpreviewfeature]: /powershell/module/az.resources/register-azproviderpreviewfeature |
| 161 | +[az-feature-list]: /cli/azure/feature#az_feature_list |
| 162 | +[get-azproviderpreviewfeature]: /powershell/module/az.resources/get-azproviderpreviewfeature |
| 163 | +[az-provider-register]: /cli/azure/provider#az_provider_register |
| 164 | +[register-azresourceprovider]: /powershell/module/az.resources/register-azresourceprovider |
| 165 | + |
| 166 | +[arm-vms]: https://azure.microsoft.com/blog/azure-virtual-machines-with-ampere-altra-arm-based-processors-generally-available/ |
| 167 | +[trivy]: https://github.com/aquasecurity/trivy |
0 commit comments