reorganizing autoscale doc

Author: hsrasheed

articles/hdinsight/hdinsight-autoscale-clusters.md

@@ -18,7 +18,14 @@ Azure HDInsight's free Autoscale feature can automatically increase or decrease
 
 The Autoscale feature uses two types of conditions to trigger scaling events: thresholds for various cluster performance metrics (called *load-based scaling*) and time-based triggers (called *schedule-based scaling*). Load-based scaling changes the number of nodes in your cluster, within a range that you set, to ensure optimal CPU usage and minimize running cost. Schedule-based scaling changes the number of nodes in your cluster based on operations that you associate with specific dates and times.
 
-### Metrics monitoring
+### Choosing load-based or schedule-based scaling
+
+Consider the following factors when choosing a scaling type:
+
+* Load variance: does the load of the cluster follow a consistent pattern at specific times, on specific days? If not, load based scheduling is a better option.
+* SLA requirements: Autoscale scaling is reactive instead of predictive. Will there be a sufficient delay between when the load starts to increase and when the cluster needs to be at its target size? If there are strict SLA requirements and the load is a fixed known pattern, 'schedule based' is a better option.
+
+### Cluster metrics
 
 Autoscale continuously monitors the cluster and collects the following metrics:
 
@@ -46,6 +53,24 @@ For scale-up, Autoscale issues a scale-up request to add the required number of
 
 For scale-down, Autoscale issues a request to remove a certain number of nodes. The scale-down is based on the number of AM containers per node. And the current CPU and memory requirements. The service also detects which nodes are candidates for removal based on current job execution. The scale down operation first decommissions the nodes, and then removes them from the cluster.
 
+### Cluster compatibility
+
+> [!Important]
+> The Azure HDInsight Autoscale feature was released for general availability on November 7th, 2019 for Spark and Hadoop clusters and included improvements not available in the preview version of the feature. If you created a Spark cluster prior to November 7th, 2019 and want to use the Autoscale feature on your cluster, the recommended path is to create a new cluster, and enable Autoscale on the new cluster.
+>
+> Autoscale for Interactive Query (LLAP) and HBase clusters is still in preview. Autoscale is only available on Spark, Hadoop, Interactive Query, and HBase clusters.
+
+The following table describes the cluster types and versions that are compatible with the Autoscale feature.
+
+| Version | Spark | Hive | LLAP | HBase | Kafka | Storm | ML |
+|---|---|---|---|---|---|---|---|
+| HDInsight 3.6 without ESP | Yes | Yes | Yes | Yes* | No | No | No |
+| HDInsight 4.0 without ESP | Yes | Yes | Yes | Yes* | No | No | No |
+| HDInsight 3.6 with ESP | Yes | Yes | Yes | Yes* | No | No | No |
+| HDInsight 4.0 with ESP | Yes | Yes | Yes | Yes* | No | No | No |
+
+\* HBase clusters can only be configured for schedule-based scaling, not load-based.
+
 ## Get started
 
 ### Create a cluster with load-based Autoscaling
@@ -94,24 +119,6 @@ Your subscription has a capacity quota for each region. The total number of core
 
 For more information on HDInsight cluster creation using the Azure portal, see [Create Linux-based clusters in HDInsight using the Azure portal](hdinsight-hadoop-create-linux-clusters-portal.md).  
 
-## Cluster compatibility
-
-> [!Important]
-> The Azure HDInsight Autoscale feature was released for general availability on November 7th, 2019 for Spark and Hadoop clusters and included improvements not available in the preview version of the feature. If you created a Spark cluster prior to November 7th, 2019 and want to use the Autoscale feature on your cluster, the recommended path is to create a new cluster, and enable Autoscale on the new cluster.
->
-> Autoscale for Interactive Query (LLAP) and HBase clusters is still in preview. Autoscale is only available on Spark, Hadoop, Interactive Query, and HBase clusters.
-
-The following table describes the cluster types and versions that are compatible with the Autoscale feature.
-
-| Version | Spark | Hive | LLAP | HBase | Kafka | Storm | ML |
-|---|---|---|---|---|---|---|---|
-| HDInsight 3.6 without ESP | Yes | Yes | Yes | Yes* | No | No | No |
-| HDInsight 4.0 without ESP | Yes | Yes | Yes | Yes* | No | No | No |
-| HDInsight 3.6 with ESP | Yes | Yes | Yes | Yes* | No | No | No |
-| HDInsight 4.0 with ESP | Yes | Yes | Yes | Yes* | No | No | No |
-
-\* HBase clusters can only be configured for schedule-based scaling, not load-based.
-
 ### Create a cluster with a Resource Manager template
 
 #### Load-based autoscaling
@@ -198,32 +205,7 @@ Use the appropriate parameters in the request payload. The json payload below co
 
 See the previous section on [enabling load-based autoscale](#load-based-autoscaling) for a full description of all payload parameters.
 
-## Guidelines
-
-### Choosing load-based or schedule-based scaling
-
-Consider the following factors before making a decision on which mode to choose:
-
-* Enable Autoscale during cluster creation.
-* The minimum number of nodes should be at least three.
-* Load variance: does the load of the cluster follow a consistent pattern at specific times, on specific days. If not, load based scheduling is a better option.
-* SLA requirements: Autoscale scaling is reactive instead of predictive. Will there be a sufficient delay between when the load starts to increase and when the cluster needs to be at its target size? If there are strict SLA requirements and the load is a fixed known pattern, 'schedule based' is a better option.
-
-### Consider the latency of scale up or scale down operations
-
-It can take 10 to 20 minutes for a scaling operation to complete. When setting up a customized schedule, plan for this delay. For example, if you need the cluster size to be 20 at 9:00 AM, set the schedule trigger to an earlier time such as 8:30 AM so that the scaling operation has completed by 9:00 AM.
-
-### Preparation for scaling down
-
-During cluster scaling down process, Autoscale will decommission the nodes to meet the target size. If tasks are running on those nodes, Autoscale will wait until the tasks are completed. Since each worker node also serves a role in HDFS, the temp data will be shifted to the remaining nodes. So you should make sure there's enough space on the remaining nodes to host all the temp data.
-
-The running jobs will continue. The pending jobs will wait for scheduling with fewer available worker nodes.
-
-### Minimum cluster size
-
-Don't scale your cluster down to fewer than three nodes. Scaling your cluster to fewer than three nodes can result in it getting stuck in safe mode because of insufficient file replication.  For more information, see [Getting stuck in safe mode](./hdinsight-scaling-best-practices.md#getting-stuck-in-safe-mode).
-
-## Monitoring
+## Monitoring Autoscale activities
 
 ### Cluster status
 
@@ -251,6 +233,22 @@ Select **Metrics** under **Monitoring**. Then select **Add metric** and **Number
 
 ![Enable worker node schedule-based autoscale metric](./media/hdinsight-autoscale-clusters/hdinsight-autoscale-clusters-chart-metric.png)
 
+## Other considerations
+
+### Consider the latency of scale up or scale down operations
+
+It can take 10 to 20 minutes for a scaling operation to complete. When setting up a customized schedule, plan for this delay. For example, if you need the cluster size to be 20 at 9:00 AM, set the schedule trigger to an earlier time such as 8:30 AM so that the scaling operation has completed by 9:00 AM.
+
+### Preparation for scaling down
+
+During cluster scaling down process, Autoscale will decommission the nodes to meet the target size. If tasks are running on those nodes, Autoscale will wait until the tasks are completed. Since each worker node also serves a role in HDFS, the temp data will be shifted to the remaining nodes. So you should make sure there's enough space on the remaining nodes to host all the temp data.
+
+The running jobs will continue. The pending jobs will wait for scheduling with fewer available worker nodes.
+
+### Minimum cluster size
+
+Don't scale your cluster down to fewer than three nodes. Scaling your cluster to fewer than three nodes can result in it getting stuck in safe mode because of insufficient file replication.  For more information, see [Getting stuck in safe mode](./hdinsight-scaling-best-practices.md#getting-stuck-in-safe-mode).
+
 ## Next steps
 
 Read about guidelines for scaling clusters manually in [Scaling guidelines](hdinsight-scaling-best-practices.md)