Skip to content

Commit 2492c8c

Browse files
Stacyrch140Stacyrch140
authored
Merge pull request #266580 from RoseHJM/ade-runner
ADE - ARM/Bicep Custom Runner how-to article
2 parents db8da88 + d1d123d commit 2492c8c

File tree

4 files changed

+254
-1
lines changed

4 files changed

+254
-1
lines changed

‎articles/deployment-environments/how-to-configure-extensibility-bicep-container-image.md

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
---
2+
title: ADE extensibility model for custom ARM and Bicep images
3+
titleSuffix: Azure Deployment Environments
4+
description: Learn how to use the ADE extensibility model to build and utilize custom ARM and Bicep images within your environment definitions for deployment environments.
5+
ms.service: deployment-environments
6+
author: RoseHJM
7+
ms.author: rosemalcolm
8+
ms.date: 04/13/2024
9+
ms.topic: how-to
10+
11+
#customer intent: As a developer, I want to learn how to build and utilize custom images within my environment definitions for deployment environments.
12+
---
13+
14+
# Configure container image to execute deployments with ARM and Bicep
15+
16+
In this article, you learn how to build and utilize custom images within your environment definitions for deployments in Azure Deployment Environments (ADE).
17+
18+
ADE supports an extensibility model that enables you to create custom images that you can use in your environment definitions. To leverage this extensibility model, you can create your own custom images, and store them in a container registry like the [Microsoft Artifact Registry](https://mcr.microsoft.com/)(also known as the Microsoft Container Registry). You can then reference these images in your environment definitions to deploy your environments.
19+
20+
The ADE team provides a selection of images to get you started, including a core image, and an Azure Resource Manager (ARM)/Bicep image. You can access these sample images in the [Runner-Images](https://github.com/Azure/deployment-environments/tree/custom-runner-private-preview/Runner-Images) folder.
21+
22+
The ADE CLI is a tool that allows you to build custom images by using ADE base images. You can use the ADE CLI to customize your deployments and deletions to fit your workflow. The ADE CLI is preinstalled on the sample images. To learn more about the ADE CLI, see the [CLI Custom Runner Image reference](https://aka.ms/deployment-environments/ade-cli-reference).
23+
24+
## Prerequisites
25+
26+
- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).
27+
28+
## Create and build a Docker image
29+
30+
In this example, you learn how to build a Docker image to utilize ADE deployments and access the ADE CLI, basing your image off of one of the ADE authored images.
31+
32+
### FROM statement
33+
34+
Include a FROM statement within a created DockerFile for your new image pointing to a sample image hosted on Microsoft Artifact Registry.
35+
36+
Here's an example FROM statement, referencing the sample core image:
37+
38+
```docker
39+
FROM mcr.microsoft.com/deployment-environments/runners/core:latest
40+
```
41+
42+
This statement pulls the most recently published core image, and makes it a basis for your custom image.
43+
44+
### Install Bicep in a Dockerfile
45+
46+
You can install the Bicep package with the Azure CLI by using the RUN statement, as shown in the following example:
47+
48+
```azure cli
49+
RUN az bicep install
50+
```
51+
52+
The ADE sample images are based on the Azure CLI image, and have the ADE CLI and JQ packages preinstalled. You can learn more about the [Azure CLI](/cli/azure/), and the [JQ package](https://devdocs.io/jq/).
53+
54+
To install any more packages you need within your image, use the RUN statement.
55+
56+
### Execute operation shell scripts
57+
58+
Within the sample images, operations are determined and executed based on the operation name. Currently, the two operation names supported are *deploy* and *delete*.
59+
60+
To set up your custom image to utilize this structure, specify a folder at the level of your Dockerfile named *scripts*, and specify two files, *deploy.sh*, and *delete.sh*. The deploy shell script runs when your environment is created or redeployed, and the delete shell script runs when your environment is deleted. You can see examples of shell scripts in the repository under the [Runner-Images folder for the ARM-Bicep](https://github.com/Azure/deployment-environments/tree/custom-runner-private-preview/Runner-Images/ARM-Bicep) image.
61+
62+
To ensure these shell scripts are executable, add the following lines to your Dockerfile:
63+
64+
```docker
65+
COPY scripts/* /scripts/
66+
RUN find /scripts/ -type f -iname "*.sh" -exec dos2unix '{}' '+'
67+
RUN find /scripts/ -type f -iname "*.sh" -exec chmod +x {} \;
68+
```
69+
70+
### Author operation shell scripts to deploy ARM or Bicep templates
71+
To ensure you can successfully deploy ARM or Bicep infrastructure through ADE, you must:
72+
- Convert ADE parameters to ARM-acceptable parameters
73+
- Resolve linked templates if they're used in the deployment
74+
- Use privileged managed identity to perform the deployment
75+
76+
During the core image's entrypoint, any parameters set for the current environment are stored under the variable `$ADE_OPERATION_PARAMETERS`. In order to convert them to ARM-acceptable parameters, you can run the following command using JQ:
77+
```bash
78+
# format the parameters as arm parameters
79+
deploymentParameters=$(echo "$ADE_OPERATION_PARAMETERS" | jq --compact-output '{ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", "contentVersion": "1.0.0.0", "parameters": (to_entries | if length == 0 then {} else (map( { (.key): { "value": .value } } ) | add) end) }' )
80+
```
81+
82+
Next, to resolve any linked templates used within an ARM JSON-based template, you can decompile the main template file, which resolves all the local infrastructure files used into many Bicep modules. Then, rebuild those modules back into a single ARM template with the linked templates embedded into the main ARM template as nested templates. This step is only necessary during the deployment operation. The main template file can be specified using the `$ADE_TEMPLATE_FILE` set during the core image's entrypoint, and you should reset this variable with the recompiled template file. See the following example:
83+
```bash
84+
if [[ $ADE_TEMPLATE_FILE == *.json ]]; then
85+
86+
hasRelativePath=$( cat $ADE_TEMPLATE_FILE | jq '[.. | objects | select(has("templateLink") and (.templateLink | has("relativePath")))] | any' )
87+
88+
if [ "$hasRelativePath" = "true" ]; then
89+
echo "Resolving linked ARM templates"
90+
91+
bicepTemplate="${ADE_TEMPLATE_FILE/.json/.bicep}"
92+
generatedTemplate="${ADE_TEMPLATE_FILE/.json/.generated.json}"
93+
94+
az bicep decompile --file "$ADE_TEMPLATE_FILE"
95+
az bicep build --file "$bicepTemplate" --outfile "$generatedTemplate"
96+
97+
# Correctly reassign ADE_TEMPLATE_FILE without the $ prefix during assignment
98+
ADE_TEMPLATE_FILE="$generatedTemplate"
99+
fi
100+
fi
101+
```
102+
To provide the permissions a deployment requires to execute the deployment and deletion of resources within the subscription, use the privileged managed identity associated with the ADE project environment type. If your deployment needs special permissions to complete, such as particular roles, assign those roles to the project environment type's identity. Sometimes, the managed identity isn't immediately available when entering the container; you can retry until the login is successful.
103+
```bash
104+
echo "Signing into Azure using MSI"
105+
while true; do
106+
# managed identity isn't available immediately
107+
# we need to do retry after a short nap
108+
az login --identity --allow-no-subscriptions --only-show-errors --output none && {
109+
echo "Successfully signed into Azure"
110+
break
111+
} || sleep 5
112+
done
113+
```
114+
115+
To begin deployment of the ARM or Bicep templates, run the `az deployment group create` command. When running this command inside the container, choose a deployment name that doesn't override any past deployments, and use the `--no-prompt true` and `--only-show-errors` flags to ensure the deployment doesn't fail on any warnings or stall on waiting for user input, as shown in the following example:
116+
117+
```bash
118+
deploymentName=$(date +"%Y-%m-%d-%H%M%S")
119+
az deployment group create --subscription $ADE_SUBSCRIPTION_ID \
120+
--resource-group "$ADE_RESOURCE_GROUP_NAME" \
121+
--name "$deploymentName" \
122+
--no-prompt true --no-wait \
123+
--template-file "$ADE_TEMPLATE_FILE" \
124+
--parameters "$deploymentParameters" \
125+
--only-show-errors
126+
```
127+
128+
To delete an environment, perform a Complete-mode deployment and provide an empty ARM template, which removes all resources within the specified ADE resource group, as shown in the following example:
129+
```bash
130+
deploymentName=$(date +"%Y-%m-%d-%H%M%S")
131+
az deployment group create --resource-group "$ADE_RESOURCE_GROUP_NAME" \
132+
--name "$deploymentName" \
133+
--no-prompt true --no-wait --mode Complete \
134+
--only-show-errors \
135+
--template-file "$DIR/empty.json"
136+
```
137+
138+
You can check the provisioning state and details by running the below commands. ADE uses some special functions to read and provide additional context based on the provisioning details, which you can find in the [Runner-Images](https://github.com/Azure/deployment-environments/tree/custom-runner-private-preview/Runner-Images) folder. A simple implementation could be as follows:
139+
```bash
140+
if [ $? -eq 0 ]; then # deployment successfully created
141+
while true; do
142+
143+
sleep 1
144+
145+
ProvisioningState=$(az deployment group show --resource-group "$ADE_RESOURCE_GROUP_NAME" --name "$deploymentName" --query "properties.provisioningState" -o tsv)
146+
ProvisioningDetails=$(az deployment operation group list --resource-group "$ADE_RESOURCE_GROUP_NAME" --name "$deploymentName")
147+
148+
echo "$ProvisioningDetails"
149+
150+
if [[ "CANCELED|FAILED|SUCCEEDED" == *"${ProvisioningState^^}"* ]]; then
151+
152+
echo -e "\nDeployment $deploymentName: $ProvisioningState"
153+
154+
if [[ "CANCELED|FAILED" == *"${ProvisioningState^^}"* ]]; then
155+
exit 11
156+
else
157+
break
158+
fi
159+
fi
160+
done
161+
fi
162+
```
163+
164+
Finally, to view the outputs of your deployment and pass them to ADE to make them accessible via the Azure CLI, you can run the following commands:
165+
```bash
166+
deploymentOutput=$(az deployment group show -g "$ADE_RESOURCE_GROUP_NAME" -n "$deploymentName" --query properties.outputs)
167+
if [ -z "$deploymentOutput" ]; then
168+
deploymentOutput="{}"
169+
fi
170+
echo "{\"outputs\": $deploymentOutput}" > $ADE_OUTPUTS
171+
```
172+
173+
174+
### Build the image
175+
176+
Before you build the image to be pushed to your registry, ensure the [Docker Engine is installed](https://docs.docker.com/desktop/) on your computer. Then, navigate to the directory of your Dockerfile, and run the following command:
177+
178+
```docker
179+
docker build . -t {YOUR_REGISTRY}.azurecr.io/{YOUR_REPOSITORY}:{YOUR_TAG}
180+
```
181+
182+
For example, if you want to save your image under a repository within your registry named `customImage`, and upload with the tag version of `1.0.0`, you would run:
183+
184+
```docker
185+
docker build . -t {YOUR_REGISTRY}.azurecr.io/customImage:1.0.0
186+
```
187+
188+
## Push the Docker image to a registry
189+
190+
In order to use custom images, you need to set up a publicly accessible image registry with anonymous image pull enabled. This way, Azure Deployment Environments can access your custom image to execute in our container.
191+
192+
Azure Container Registry is an Azure offering that stores container images and similar artifacts.
193+
194+
To create a registry, which can be done through the Azure CLI, the Azure portal, PowerShell commands, and more, follow one of the [quickstarts](/azure/container-registry/container-registry-get-started-azure-cli).
195+
196+
To set up your registry to have anonymous image pull enabled, run the following commands in the Azure CLI:
197+
198+
```azurecli
199+
az login
200+
az acr login -n {YOUR_REGISTRY}
201+
az acr update -n {YOUR_REGISTRY} --public-network-enabled true
202+
az acr update -n {YOUR_REGISTRY} --anonymous-pull-enabled true
203+
```
204+
205+
When you're ready to push your image to your registry, run the following command:
206+
207+
```docker
208+
docker push {YOUR_REGISTRY}.azurecr.io/{YOUR_IMAGE_LOCATION}:{YOUR_TAG}
209+
```
210+
211+
## Connect the image to your environment definition
212+
213+
When authoring environment definitions to use your custom image in their deployment, edit the `runner` property on the manifest file (environment.yaml or manifest.yaml).
214+
215+
```yaml
216+
runner: "{YOUR_REGISTRY}.azurecr.io/{YOUR_REPOSITORY}:{YOUR_TAG}"
217+
```
218+
219+
## Access operation logs and error details
220+
221+
ADE stores error details for a failed deployment the *$ADE_ERROR_LOG* file.
222+
223+
To troubleshoot a failed deployment:
224+
225+
1. Sign in to the [Developer Portal](https://devportal.microsoft.com/).
226+
1. Identify the environment that failed to deploy, and select **See details**.
227+
228+
:::image type="content" source="media/how-to-configure-extensibility-bicep-container-image/failed-deployment-card.png" alt-text="Screenshot showing failed deployment error details, specifically an invalid name for a storage account." lightbox="media/how-to-configure-extensibility-bicep-container-image/failed-deployment-card.png":::
229+
230+
1. Review the error details in the **Error Details** section.
231+
232+
:::image type="content" source="media/how-to-configure-extensibility-bicep-container-image/deployment-error-details.png" alt-text="Screenshot showing a failed deployment of an environment with the See Details button displayed." lightbox="media/how-to-configure-extensibility-bicep-container-image/deployment-error-details.png":::
233+
234+
Additionally, you can use the Azure CLI to view an environment's error details using the following command:
235+
```bash
236+
az devcenter dev environment show --environment-name {YOUR_ENVIRONMENT_NAME} --project {YOUR_PROJECT_NAME}
237+
```
238+
239+
To view the operation logs for an environment deployment or deletion, use the Azure CLI to retrieve the latest operation for your environment, and then view the logs for that operation ID.
240+
241+
```bash
242+
# Get list of operations on the environment, choose the latest operation
243+
az devcenter dev environment list-operation --environment-name {YOUR_ENVIRONMENT_NAME} --project {YOUR_PROJECT_NAME}
244+
# Using the latest operation ID, view the operation logs
245+
az devcenter dev environment show-logs-by-operation --environment-name {YOUR_ENVIRONMENT_NAME} --project {YOUR_PROJECT_NAME} --operation-id {LATEST_OPERATION_ID}
246+
```
247+
248+
## Related content
249+
250+
- [ADE CLI Custom Runner Image reference](https://aka.ms/deployment-environments/ade-cli-reference)

‎articles/deployment-environments/media/how-to-configure-extensibility-bicep-container-image/deployment-error-details.png

161 KB
Loading

‎articles/deployment-environments/media/how-to-configure-extensibility-bicep-container-image/failed-deployment-card.png

47 KB
Loading

‎articles/deployment-environments/toc.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,11 +61,14 @@ items:
6161
- name: Configure container image to execute deployments
6262
href: how-to-configure-extensibility-generic-container-image.md
6363
displayName: custom image, custom runner, extensibility
64+
- name: Configure ARM/Bicep container image to execute deployments
65+
href: how-to-configure-extensibility-bicep-container-image.md
66+
displayName: custom image, custom runner, extensibility
6467
- name: Create and manage environments
6568
expanded: false
6669
items:
6770
- name: Create environments with Azure Developer CLI (AZD)
68-
href: how-to-create-environment-with-azure-developer.md
71+
href: how-to-create-environment-with-azure-developer.md
6972
- name: Manage environments in the developer portal
7073
href: how-to-manage-environments.md
7174
- name: Create environments with Azure CLI

0 commit comments

Comments
 (0)