You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With this autoscale formula, the pool is initially created with a single VM. The `$PendingTasks` metric defines the number of tasks that are running or queued. The formula finds the average number of pending tasks in the last 180 seconds and sets the `$TargetDedicatedNodes` variable accordingly. The formula ensures that the target number of dedicated nodes never exceeds 25 VMs. As new tasks are submitted, the pool automatically grows. As tasks complete, VMs become free one by one and the autoscaling formula shrinks the pool.
@@ -79,6 +80,7 @@ This formula scales dedicated nodes, but can be modified to apply to scale low-p
This example creates a pool that starts with 25 low-priority nodes. Every time a low-priority node is preempted, it is replaced with a dedicated node. As with the first example, the `maxNumberofVMs` variable prevents the pool from exceeding 25 VMs. This example is useful for taking advantage of low-priority VMs while also ensuring that only a fixed number of preemptions will occur for the lifetime of the pool.
@@ -100,7 +102,7 @@ You can get and set the values of these service-defined variables to manage the
100
102
| --- | --- |
101
103
| $TargetDedicatedNodes |The target number of dedicated compute nodes for the pool. The number of dedicated nodes is specified as a target because a pool may not always achieve the desired number of nodes. For example, if the target number of dedicated nodes is modified by an autoscale evaluation before the pool has reached the initial target, then the pool may not reach the target. <br /><br /> A pool in an account created with the Batch Service configuration may not achieve its target if the target exceeds a Batch account node or core quota. A pool in an account created with the User Subscription configuration may not achieve its target if the target exceeds the shared core quota for the subscription.|
102
104
| $TargetLowPriorityNodes |The target number of low-priority compute nodes for the pool. The number of low-priority nodes is specified as a target because a pool may not always achieve the desired number of nodes. For example, if the target number of low-priority nodes is modified by an autoscale evaluation before the pool has reached the initial target, then the pool may not reach the target. A pool may also not achieve its target if the target exceeds a Batch account node or core quota. <br /><br /> For more information on low-priority compute nodes, see [Use low-priority VMs with Batch (Preview)](batch-low-pri-vms.md). |
103
-
| $NodeDeallocationOption |The action that occurs when compute nodes are removed from a pool. Possible values are:<ul><li>**requeue**-- The default value. Terminates tasks immediately and puts them back on the job queue so that they are rescheduled.<li>**terminate**--Terminates tasks immediately and removes them from the job queue.<li>**taskcompletion**--Waits for currently running tasks to finish and then removes the node from the pool. This is the **recommended value** in most cases. <li>**retaineddata**--Waits for all the local task-retained data on the node to be cleaned up before removing the node from the pool.</ul> |
105
+
| $NodeDeallocationOption |The action that occurs when compute nodes are removed from a pool. Possible values are:<ul><li>**requeue**-- The default value. Terminates tasks immediately and puts them back on the job queue so that they are rescheduled. This action ensures the target number of nodes is reach as quickly as possible, but may be less efficient, as any running tasks will be interrupted and have to be restarted, wasting any work they had already done. <li>**terminate**--Terminates tasks immediately and removes them from the job queue.<li>**taskcompletion**--Waits for currently running tasks to finish and then removes the node from the pool. Use this option to avoid tasks being interrupted and requeued, wasting any work the task has done. <li>**retaineddata**--Waits for all the local task-retained data on the node to be cleaned up before removing the node from the pool.</ul> |
104
106
105
107
You can get the value of these service-defined variables to make adjustments that are based on metrics from the Batch service:
106
108
@@ -331,8 +333,9 @@ You build an autoscale formula by forming statements that use the above componen
331
333
First, let's define the requirements for our new autoscale formula. The formula should:
332
334
333
335
1. Increase the target number of dedicated compute nodes in a pool if CPU usage is high.
334
-
2. Decrease the target number of dedicated compute nodes in a pool when CPU usage is low.
335
-
3. Always restrict the maximum number of dedicated nodes to 400.
336
+
1. Decrease the target number of dedicated compute nodes in a pool when CPU usage is low.
337
+
1. Always restrict the maximum number of dedicated nodes to 400.
338
+
1. When reducing the number of nodes, do not remove nodes that are running tasks; if necessary, wait until tasks have finished to remove nodes.
336
339
337
340
To increase the number of nodes during high CPU usage, define the statement that populates a user-defined variable (`$totalDedicatedNodes`) with a value that is 110 percent of the current target number of dedicated nodes, but only if the minimum average CPU usage during the last 10 minutes was above 70 percent. Otherwise, use the value for the current number of dedicated nodes.
// Set node deallocation mode - keep nodes active only until tasks finish
679
+
// Set node deallocation mode - let running tasks finish before removing a node
676
680
$NodeDeallocationOption=taskcompletion;
677
681
```
678
682
679
-
> [!NOTE]
680
-
> It's highly recommended to use `$NodeDeallocationOption = taskcompletion`, which waits for currently running tasks to finish, and then removes the node from the pool. If you use the default value, `$NodeDeallocationOption = requeue`, the task terminates immediately, wasting what has been run so far, and puts nodes back on the job queue so that they are rescheduled. SFor more information, see the [Variables](#variables) table.
681
-
682
683
### Example 3: Accounting for parallel tasks
683
684
684
685
This example adjusts the pool size based on the number of tasks. This formula also takes into account the [MaxTasksPerComputeNode][net_maxtasks] value that has been set for the pool. This approach is useful in situations where [parallel task execution](batch-parallel-node-tasks.md) has been enabled on your pool.
0 commit comments