Merge pull request #279558 from alfpark/alpark/batch

Update Batch autoscale and best practices docs

Author: prmerger-automator[bot]

articles/batch/batch-automatic-scaling.md

@@ -2,7 +2,7 @@
 title: Autoscale compute nodes in an Azure Batch pool
 description: Enable automatic scaling on an Azure Batch cloud pool to dynamically adjust the number of compute nodes in the pool.
 ms.topic: how-to
-ms.date: 06/11/2024
+ms.date: 06/27/2024
 ms.custom: H1Hack27Feb2017, fasttrack-edit, devx-track-csharp
 ---
 
@@ -124,11 +124,6 @@ You can get the value of these service-defined variables to make adjustments tha
 | $UsableNodeCount | The number of usable compute nodes. |
 | $PreemptedNodeCount | The number of nodes in the pool that are in a preempted state. |
 
-> [!WARNING]
-> Select service-defined variables will be retired after **31 March 2024** as noted in the table above. After the retirement
-> date, these service-defined variables will no longer be populated with sample data. Please discontinue use of these variables
-> before this date.
-
 > [!NOTE]
 > Use `$RunningTasks` when scaling based on the number of tasks running at a point in time, and `$ActiveTasks` when scaling based on the number of tasks that are queued up to run.
 
@@ -597,7 +592,7 @@ In Batch .NET, the [CloudPool.AutoScaleRun](/dotnet/api/microsoft.azure.batch.cl
 - [AutoScaleRun.Results](/dotnet/api/microsoft.azure.batch.autoscalerun.results)
 - [AutoScaleRun.Error](/dotnet/api/microsoft.azure.batch.autoscalerun.error)
 
-In the REST API, the [Get information about a pool request](/rest/api/batchservice/get-information-about-a-pool) returns information about the pool, which includes the latest automatic scaling run information in the [autoScaleRun](/rest/api/batchservice/get-information-about-a-pool) property.
+In the REST API, [information about a pool](/rest/api/batchservice/get-information-about-a-pool) includes the latest automatic scaling run information in the [autoScaleRun](/rest/api/batchservice/get-information-about-a-pool) property.
 
 The following C# example uses the Batch .NET library to print information about the last autoscaling run on pool *myPool*.
 

articles/batch/best-practices.md

@@ -1,7 +1,7 @@
 ---
 title: Best practices
 description: Learn best practices and useful tips for developing your Azure Batch solutions.
-ms.date: 05/31/2024
+ms.date: 06/27/2024
 ms.topic: conceptual
 ---
 
@@ -20,15 +20,6 @@ This article discusses best practices and useful tips for using the Azure Batch
 
 - **Pool allocation mode:** When creating a Batch account, you can choose between two pool allocation modes: **Batch service** or **user subscription**. For most cases, you should use the default Batch service mode, in which pools are allocated behind the scenes in Batch-managed subscriptions. In the alternative user subscription mode, Batch VMs and other resources are created directly in your subscription when a pool is created. User subscription accounts are primarily used to enable a small but important subset of scenarios. For more information, see [configuration for user subscription mode](batch-account-create-portal.md#additional-configuration-for-user-subscription-mode).
 
-- **`virtualMachineConfiguration` or `cloudServiceConfiguration`:** While you can currently create pools using either
-configuration, new pools should be configured using `virtualMachineConfiguration` and not `cloudServiceConfiguration`.
-All current and new Batch features will be supported by Virtual Machine Configuration pools. Cloud Service Configuration
-pools don't support all features and no new capabilities are planned. You won't be able to create new
-`cloudServiceConfiguration` pools or add new nodes to existing pools
-[after February 29, 2024](https://azure.microsoft.com/updates/azure-batch-cloudserviceconfiguration-pools-will-be-retired-on-29-february-2024/).
-For more information, see
-[Migrate Batch pool configuration from Cloud Services to Virtual Machine](batch-pool-cloud-service-to-virtual-machine-configuration.md).
-
 - **`classic` or `simplified` node communication mode:** Pools can be configured in one of two node communication modes,
 classic or [simplified](simplified-compute-node-communication.md). In the classic node communication model, the Batch service
 initiates communication to the compute nodes, and compute nodes also require communicating to Azure Storage. In the simplified
@@ -86,10 +77,17 @@ Before you recreate or resize your pool, you should download any node agent logs
 #### Operating system updates
 
 It's recommended that the VM image selected for a Batch pool should be up-to-date with the latest publisher provided security updates.
-Some images may perform automatic updates upon boot (or shortly thereafter), which may interfere with certain user directed actions such
+Some images may perform automatic package updates upon boot (or shortly thereafter), which may interfere with certain user directed actions such
 as retrieving package repository updates (for example, `apt update`) or installing packages during actions such as a
 [StartTask](jobs-and-tasks.md#start-task).
 
+It's recommended to enable [Auto OS upgrade for Batch pools](batch-upgrade-policy.md), which allows the underlying
+Azure infrastructure to coordinate updates across the pool. This option can be configured to be nondisrupting for task
+execution. Automatic OS upgrade doesn't support all operating systems that Batch supports. For more information, see the
+[Virtual Machine Scale Sets Auto OS upgrade Support Matrix](../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md#supported-os-images).
+For Windows operating systems, ensure that you aren't enabling the property
+`virtualMachineConfiguration.windowsConfiguration.enableAutomaticUpdates` when using Auto OS upgrade on the Batch pool.
+
 Azure Batch doesn't verify or guarantee that images allowed for use with the service have the latest security updates.
 Updates to images are under the purview of the publisher of the image, and not that of Azure Batch. For certain images published
 under `microsoft-azure-batch`, there's no guarantee that these images are kept up-to-date with their upstream derived image.
@@ -122,7 +120,7 @@ Pools can be created using third-party images published to Azure Marketplace. Wi
 
 ### Container pools
 
-When specifying a Batch pool with a [virtual network](batch-virtual-network.md), there can be interaction
+When you create a Batch pool with a [virtual network](batch-virtual-network.md), there can be interaction
 side effects between the specified virtual network and the default Docker bridge. Docker, by default, will
 create a network bridge with a subnet specification of `172.17.0.0/16`. Ensure that there are no conflicting
 IP ranges between the Docker network bridge and your virtual network.
@@ -215,7 +213,7 @@ Tasks that only run for one to two seconds aren't ideal. Try to do a significant
 
 ### Use pool scope for short tasks on Windows nodes
 
-When scheduling a task on Batch nodes, you can choose whether to run it with task scope or pool scope. If the task will only run for a short time, task scope can be inefficient due to the resources needed to create the auto-user account for that task. For greater efficiency, consider setting these tasks to pool scope. For more information, see [Run a task as an auto-user with pool scope](batch-user-accounts.md#run-a-task-as-an-auto-user-with-pool-scope).
+When scheduling a task on Batch nodes, you can choose whether to run it with task scope or pool scope. If the task will only run for a short time, task scope can be inefficient due to the resources needed to create the autouser account for that task. For greater efficiency, consider setting these tasks to pool scope. For more information, see [Run a task as an autouser with pool scope](batch-user-accounts.md#run-a-task-as-an-auto-user-with-pool-scope).
 
 ## Nodes
 
@@ -322,12 +320,6 @@ promotion into production use.
 
 If you notice a problem involving the behavior of a node or tasks running on a node, collect the Batch agent logs prior to deallocating the nodes in question. The Batch agent logs can be collected using the Upload Batch service logs API. These logs can be supplied as part of a support ticket to Microsoft and will help with issue troubleshooting and resolution.
 
-### Manage OS upgrades
-
-For user subscription mode Batch accounts, automated OS upgrades can interrupt task progress, especially if the tasks are long-running. [Building idempotent tasks](#build-durable-tasks) can help to reduce errors caused by these interruptions. We also recommend [scheduling OS image upgrades for times when tasks aren't expected to run](../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md#manually-trigger-os-image-upgrades).
-
-For Windows pools, `enableAutomaticUpdates` is set to `true` by default. Allowing automatic updates is recommended, but you can set this value to `false` if you need to ensure that an OS update doesn't happen unexpectedly.
-
 ## Batch API
 
 ### Timeout Failures
@@ -360,10 +352,6 @@ Ensure that your Batch service clients have appropriate retry policies in place
 
 Typically, virtual machines in a Batch pool are accessed through public IP addresses that can change over the lifetime of the pool. This dynamic nature can make it difficult to interact with a database or other external service that limits access to certain IP addresses. To address this concern, you can create a pool using a set of static public IP addresses that you control. For more information, see [Create an Azure Batch pool with specified public IP addresses](create-pool-public-ip.md).
 
-### Testing connectivity with Cloud Services configuration
-
-You can't use the normal "ping"/ICMP protocol with cloud services, because the ICMP protocol isn't permitted through the Azure load balancer. For more information, see [Connectivity and networking for Azure Cloud Services](../cloud-services/cloud-services-connectivity-and-networking-faq.yml#can-i-ping-a-cloud-service-).
-
 ## Batch node underlying dependencies
 
 Consider the following dependencies and restrictions when designing your Batch solutions.

articles/batch/security-best-practices.md

@@ -1,15 +1,18 @@
 ---
 title: Batch security and compliance best practices
 description: Learn best practices and useful tips for enhancing security with your Azure Batch solutions.
-ms.date: 09/13/2023
+ms.date: 06/27/2024
 ms.topic: conceptual
 ---
 
 # Batch security and compliance best practices
 
 This article provides guidance and best practices for enhancing security when using Azure Batch.
 
-By default, Azure Batch accounts have a public endpoint and are publicly accessible. When an Azure Batch pool is created, the pool is provisioned in a specified subnet of an Azure virtual network. Virtual machines in the Batch pool are accessed through public IP addresses that are created by Batch. Compute nodes in a pool can communicate with each other when needed, such as to run multi-instance tasks, but nodes in a pool can't communicate with virtual machines outside of the pool.
+By default, Azure Batch accounts have a public endpoint and are publicly accessible. When an Azure Batch pool is created,
+the pool is provisioned in a specified subnet of an Azure virtual network. Virtual machines in the Batch pool are accessed,
+by default, through public IP addresses that Batch creates. Compute nodes in a pool can communicate with each other when needed,
+such as to run multi-instance tasks, but nodes in a pool can't communicate with virtual machines outside of the pool.
 
 :::image type="content" source="media/security-best-practices/typical-environment.png" alt-text="Diagram showing a typical Batch environment.":::
 
@@ -21,9 +24,7 @@ Many features are available to help you create a more secure Azure Batch deploym
 
 ### Pool configuration
 
-Many security features are only available for pools configured using [Virtual Machine Configuration](nodes-and-pools.md#configurations), and not for pools with Cloud Services Configuration. We recommend using Virtual Machine Configuration pools, which utilize [Virtual Machine Scale Sets](../virtual-machine-scale-sets/overview.md), whenever possible.
-
-Pools can also be configured in one of two node communication modes, classic or [simplified](simplified-compute-node-communication.md).
+Pools can be configured in one of two node communication modes, classic or [simplified](simplified-compute-node-communication.md).
 In the classic node communication model, the Batch service initiates communication to the compute nodes, and compute nodes
 also require communicating to Azure Storage. In the simplified node communication model, compute nodes initiate communication
 with the Batch service. Due to the reduced scope of inbound/outbound connections required, and not requiring Azure Storage
@@ -35,14 +36,14 @@ node communication model will be
 
 Batch account access supports two methods of authentication: Shared Key and [Microsoft Entra ID](batch-aad-auth.md).
 
-We strongly recommend using Microsoft Entra ID for Batch account authentication. Some Batch capabilities require this method of authentication, including many of the security-related features discussed here. The service API authentication mechanism for a Batch account can be restricted to only Microsoft Entra ID using the [allowedAuthenticationModes](/rest/api/batchmanagement/batch-account/create) property. When this property is set, API calls using Shared Key authentication will be rejected.
+We strongly recommend using Microsoft Entra ID for Batch account authentication. Some Batch capabilities require this method of authentication, including many of the security-related features discussed here. The service API authentication mechanism for a Batch account can be restricted to only Microsoft Entra ID using the [allowedAuthenticationModes](/rest/api/batchmanagement/batch-account/create) property. When this property is set, API calls using Shared Key authentication is rejected.
 
 ### Batch account pool allocation mode
 
 When creating a Batch account, you can choose between two [pool allocation modes](accounts.md#batch-accounts):
 
-- **Batch service**: The default option, where the underlying Cloud Service or Virtual Machine Scale Set resources used to allocate and manage pool nodes are created on Batch-owned subscriptions, and aren't directly visible in the Azure portal. Only the Batch pools and nodes are visible.
-- **User subscription**: The underlying Cloud Service or Virtual Machine Scale Set resources are created in the same subscription as the Batch account. These resources are therefore visible in the subscription, in addition to the corresponding Batch resources.
+- **Batch service**: The default option, where the underlying Virtual Machine Scale Set resources used to allocate and manage pool nodes are created on Batch-owned subscriptions, and aren't directly visible in the Azure portal. Only the Batch pools and nodes are visible.
+- **User subscription**: The underlying Virtual Machine Scale Set resources are created in the same subscription as the Batch account. These resources are therefore visible in the subscription, in addition to the corresponding Batch resources.
 
 With user subscription mode, Batch VMs and other resources are created directly in your subscription when a pool is created. User subscription mode is required if you want to create Batch pools using Azure Reserved VM Instances, use Azure Policy on Virtual Machine Scale Set resources, and/or  manage the core quota on the subscription (shared across all Batch accounts in the subscription). To create a Batch account in user subscription mode, you must also register your subscription with Azure Batch, and associate the account with an Azure Key Vault.
 
@@ -72,6 +73,13 @@ Batch supports both Linux and Windows operating systems. Batch supports Linux wi
 distributions. It's recommended that the operating system is kept up-to-date with the latest patches provided by the OS
 publisher.
 
+It's recommended to enable [Auto OS upgrade for Batch pools](batch-upgrade-policy.md), which allows the underlying
+Azure infrastructure to coordinate updates across the pool. This option can be configured to be nondisrupting for task
+execution. Automatic OS upgrade doesn't support all operating systems that Batch supports. For more information, see the
+[Virtual Machine Scale Sets Auto OS upgrade Support Matrix](../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md#supported-os-images).
+For Windows operating systems, ensure that you aren't enabling the property
+`virtualMachineConfiguration.windowsConfiguration.enableAutomaticUpdates` when using Auto OS upgrade on the Batch pool.
+
 Batch support for images and node agents phase out over time, typically aligned with publisher support timelines. It's
 recommended to avoid using images with impending end-of-life (EOL) dates or images that are past their EOL date.
 It's your responsibility to periodically refresh your view of the EOL dates pertinent to your pools and migrate your workloads
@@ -89,7 +97,7 @@ at any time. EOL dates can be discovered via the
 The Batch node agent doesn't modify operating system level defaults for SSL/TLS versions or cipher suite ordering. In Windows,
 SSL/TLS versions and cipher suite order is controlled at the operating system level, and therefore the Batch node agent adopts
 the settings set by the image used by each compute node. Although the Batch node agent attempts to utilize the
-most secure settings available when possible, it can still be limited by operating system level settings.  We recommend that
+most secure settings available when possible, it can still be limited by operating system level settings. We recommend that
 you review your OS level defaults and set them appropriately for the most secure mode that is amenable for your workflow and
 organizational requirements. For more information, please visit
 [Manage TLS](/windows-server/security/tls/manage-tls) for cipher suite order enforcement and