Merge pull request #274134 from sumit-kalra/aibbyoacisubnet

AIB: Documentation for BYO-ACI-Subnet field that is introduced in API version 2024-02-01

Author: JamesJBarnett

articles/virtual-machines/TOC.yml

@@ -699,7 +699,7 @@
           href: ./windows/image-builder-virtual-desktop.md
       - name: Security
         items:
-        - name: Security mechanisms
+        - name: Isolated image builds
           displayName: Image builder, images, building
           href: ./security-isolated-image-builds-image-builder.md
         - name: Security controls by Azure Policy

articles/virtual-machines/image-builder-best-practices.md

@@ -20,6 +20,7 @@ This article describes best practices to be followed while using Azure VM Image
 - Make sure your image templates are set up for disaster recovery by following [reliability recommendation for AIB](../reliability/reliability-image-builder.md?toc=/azure/virtual-machines/toc.json&bc=/azure/virtual-machines/breadcrumb/toc.json).
 - Set up AIB [triggers](image-builder-triggers-how-to.md) to automatically rebuild your images and keep them updated.
 - Enable [VM Boot Optimization](vm-boot-optimization.md) in AIB to improve the create time for your VMs.
+- Specify your own Build VM and ACI subnets for a tighter control over deployment of networking related resource by AIB in your subscription. Specifying these subnets also leads to faster image build times. See [template reference](./linux/image-builder-json.md#vnetconfig-optional) to learn more about specifying these options.
 - Follow the [principle of least privilege](/entra/identity-platform/secure-least-privileged-access) for your AIB resources.
   - **Image Template**: A principal that has access to your image template is able to run, delete, or tamper with it. Having this access, in turn, allows the principal to change the images created by that image template.
   - **Staging Resource Group**: AIB uses a staging resource group in your subscription to customize your VM image. You must consider this resource group as sensitive and restrict access to this resource group only to required principals. Since the process of customizing your image takes place in this resource group, a principal with access to the resource group is able to compromise the image building process - for example, by injecting malware into the image. AIB also delegates privileges associated with the Template identity and Build VM identity to resources in this resource group. Hence, a principal with access to the resource group is able to get access to these identities. Further, AIB maintains a copy of your customizer artifacts in this resource group. Hence, a principal with access to the resource group is able to inspect these copies.

articles/virtual-machines/linux/image-builder-json.md

@@ -44,7 +44,8 @@ The basic format is:
       "vmSize": "<vmSize>",
       "osDiskSizeGB": <sizeInGB>,
       "vnetConfig": {
-        "subnetId": "/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName>",
+        "subnetId": "/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName1>",
+        "containerInstanceSubnetId": "/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName2>",
         "proxyVmSize": "<vmSize>"
       },
       "userAssignedIdentities": [
@@ -80,7 +81,8 @@ resource azureImageBuilder 'Microsoft.VirtualMachineImages/imageTemplates@2022-0
       vmSize: '<vmSize>'
       osDiskSizeGB: <sizeInGB>
       vnetConfig: {
-        subnetId: '/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName>'
+        subnetId: '/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName1>'
+        containerInstanceSubnetId: '/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName2>'
         proxyVmSize: '<vmSize>'
       }
       userAssignedIdentities: [
@@ -1704,18 +1706,46 @@ If you don't specify any VNet properties, Image Builder creates its own VNet, Pu
 
 ```json
 "vnetConfig": {
-  "subnetId": "/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName>"
+  "subnetId": "/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName1>",
+  "containerInstanceSubnetId": "/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName2>",
+  "proxyVmSize": "<vmSize>"
 }
 ```
 
 # [Bicep](#tab/bicep)
 
 ```bicep
 vnetConfig: {
-  subnetId: '/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName>'
+  subnetId: '/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName1>'
+  containerInstanceSubnetId: '/subscriptions/<subscriptionID>/resourceGroups/<vnetRgName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName2>'
+  proxyVmSize: '<vmSize>'
 }
 ```
 
+#### subnetId
+Resource ID of a pre-existing subnet on which the build VM and validation VM is deployed.
+
+#### containerInstanceSubnetId (optional)
+Resource ID of a pre-existing subnet on which Azure Container Instance (ACI) is deployed for [Isolated Builds](../security-isolated-image-builds-image-builder.md). If this field isn't specified, then a temporary Virtual Network,  along with subnets and Network Security Groups, is deployed in the staging resource group in addition to other networking resources (Private Endpoint, Private Link Service, Azure Load Balancer, and the Proxy VM) to enable communication between the ACI and the build VM.
+
+*[This property is only available in API versions `2024-02-01` or newer though existing templates created using earlier API versions can be updated to specify this property.]*
+
+This field can be specified only if `subnetId` is also specified and must meet the following requirements:
+- This subnet must be on the same Virtual Network as the subnet specified in `subnetId`.
+- This subnet must not be the same subnet as the one specified in `subnetId`.
+- This subnet must be delegated to the ACI service so that it can be used to deploy ACI resources. You can read more about subnet delegation for Azure services [here](../../virtual-network/manage-subnet-delegation.md). ACI specific subnet delegation information is available [here](../../container-instances/container-instances-virtual-network-concepts.md).
+- This subnet must allow outbound access to the Internet and to the subnet specified in `subnetId`. These accesses are required so that the ACI can be provisioned and it can communicate with the build VM to perform customizations/validations. On the other end, the subnet specified in `subnetId` must allow inbound access from this subnet. In general, [default security rules of Azure Network Security Groups (NSGs)](../../virtual-network/network-security-groups-overview.md#default-security-rules) allow these accesses. However, if you add more security rules to your NSGs then the following accesses must still be allowed:
+   1. Outbound access from the subnet specified in `containerInstanceSubnetId` to:
+      1. To the Internet on port 443 (*for provisioning the container image*).
+      1. To the Internet on port 445 (*for mounting file share from Azure Storage*).
+      1. To the subnet specified in `subnetId` on port 22 (for ssh/Linux) and port 5986 (for WinRM/Windows) (*for connecting to the build VM*).
+   1. Inbound access to the subnet specified in `subnetId`:
+      1. To Port 22 (for ssh/Linux) and Port 5986 (for WinRM/Windows) from the subnet specified in `containerInstanceSubnetId` (*for ACI to connect to the build VM*).
+- The [template identity](./image-builder-json.md#user-assigned-identity-for-azure-image-builder-image-template-resource) must have permission to perform 'Microsoft.Network/virtualNetworks/subnets/join/action' action on this subnet's scope. You can read more about Azure permissions for Networking [here](/azure/role-based-access-control/permissions/networking).
+
+#### proxyVmSize (optional)
+Size of the proxy virtual machine used to pass traffic to the build VM and validation VM. This field must not be specified if `containerInstanceSubnetId` is specified because no proxy virtual machine is deployed in that case. Omit or specify empty string to use the default (Standard_A1_v2).
+
 ---
 
 ## Image Template Operations

articles/virtual-machines/linux/image-builder-troubleshoot.md

@@ -22,7 +22,7 @@ Use this article to troubleshoot and resolve common issues that you might encoun
 When you're creating a build, do the following:
 
 - The VM Image Builder service communicates to the build VM by using WinRM or Secure Shell (SSH). Don't* disable these settings as part of the build.
-- VM Image Builder creates resources in the staging resource group as part of the builds. Be sure to verify that Azure Policy doesn't prevent VM Image Builder from creating or using necessary resources.
+- VM Image Builder creates resources in the staging resource group as part of the builds. The exact list of resources depends on the [networking configuration](./image-builder-json.md#vnetconfig-optional) specified in the image template. Be sure to verify that Azure Policy doesn't prevent VM Image Builder from creating or using necessary resources.
   - Create an IT_ resource group.
   - Create a storage account without a firewall.
   - Deploy [Azure Container Instances](../../container-instances/container-instances-overview.md).
@@ -800,7 +800,7 @@ Azure Image Builder builds can fail for reasons listed elsewhere in this documen
 
 #### Solution
 If you determine that a build is failing due to Isolated Image Builds, you can do the following:
-- Ensure there's no [Azure Policy](../../governance/policy/overview.md) blocking the deployment of resources mentioned in the Prerequisites section, specifically Azure Container Instances, Azure Virtual Networks, and Azure Private Endpoints.
+- Ensure there's no [Azure Policy](../../governance/policy/overview.md) blocking the deployment of resources mentioned in the [Prerequisites section](./image-builder-troubleshoot.md#prerequisites), specifically Azure Container Instances.
 - Ensure your subscription has sufficient quota of Azure Container Instances to support all your concurrent image builds. For more information, see, Azure Container Instances [quota exceeded](./image-builder-troubleshoot.md#azure-container-instances-quota-exceeded).
 
 Azure Image Builder is currently in the process of deploying Isolated Image Builds. Specific image templates are not tied to Isolated Image Builds and the same image template might or might not utilize Isolated Image Builds during different builds. You can do the following to temporarily run your build without Isolated Image Builds.