Draft of Oren's changes to existing article

Author: johnmarco

articles/openshift/howto-large-clusters.md

@@ -5,56 +5,75 @@ author: johnmarco
 ms.author: johnmarc
 ms.service: azure-redhat-openshift
 ms.topic: article
-ms.date: 08/15/2024
+ms.date: 09/12/2024
 ---
 # Deploy a large Azure Red Hat OpenShift cluster
 
-This article provides the steps and best practices for deploying large scale Azure Red Hat OpenShift clusters up to 250 nodes. For clusters of that size, a combination of control plane nodes and infrastructure nodes is needed to ensure the cluster functions properly is recommended.
+This article provides the steps and best practices for deploying large scale Azure Red Hat OpenShift clusters up to 250 worker nodes. For clusters of that size, there are some recommendations regarding control plane nodes and infrastructure nodes.
 
 > [!CAUTION]
-> Before deleting a large cluster, descale the cluster to 120 nodes or below.
+> Before deleting a cluster with more than 120 nodes, please scale down the cluster to 120 nodes or less.
 > 
 
-## Deploy a cluster
+## Recommendations
+
+### Control plane nodes
 
-For clusters with over 101 control plane nodes, use the following [virtual machine instance types](support-policies-v4.md#supported-virtual-machine-sizes) size recommendations (or similar, newer generation instance types):
+For clusters with over 100 worker nodes, the following [virtual machine instance types](support-policies-v4.md#supported-virtual-machine-sizes) (or similar, newer generation instance types) are recommended for control plane nodes:
 
 - Standard_D32s_v3
 - Standard_D32s_v4
 - Standard_D32s_v5
 
-Following is a sample script using Azure CLI to deploy a cluster with Standard_D32s_v5 as the control plane node:
-
-```azurecli
-#az aro create \ --resource-group $RESOURCEGROUP \ --name $CLUSTER \ --vnet aro-vnet \ --master-subnet master-subnet \ --worker-subnet worker-subnet --master-vm-size Standard_D32s_v5
-```
-
-## Deploy infrastructure nodes for the cluster
+### Infrastructure nodes
 
-For clusters with over 101 nodes, infrastructure nodes are required to separate cluster workloads (such as prometheus) to minimize contention with other workloads.
- 
-> [!NOTE]
-> It's recommended that you deploy three (3) infrastructure nodes per cluster for redundancy and scalability needs. 
-> 
+For clusters with over 100 worker nodes, infrastructure nodes are required to separate cluster workloads (such as prometheus) to minimize contention with other workloads. You should deploy three (3) infrastructure nodes per cluster for redundancy and scalability needs.
 
 The following instance types are recommended for infrastructure nodes:
 
 - Standard_E16as_v5
 - Standard_E16s_v5
 
-For instructions on configuring infrastructure nodes, see [Deploy infrastructure nodes in an Azure Red Hat OpenShift cluster](howto-infrastructure-nodes.md).
+For instructions on configuring infrastructure nodes, see [Deploy infrastructure nodes in an Azure Red Hat OpenShift cluster](howto-infrastructure-nodes.md). This will be configured after cluster deployment.
+
+### Add IP addresses to the load balancer
+
+Azure Red Hat OpenShift public clusters are created with a public load balancer that's used for outbound connectivity from inside the cluster. By default, one public IP address is configured on that public load balancer, and that limits the maximum node count of your cluster to 62. To be able to scale your cluster to the maximum supported number of 250 nodes, you need to assign multiple additional public IP addresses to the load balancer. You can configure up to 20 IP addresses per cluster. The outbound rules and frontend IP configurations are adjusted to accommodate the number of IP addresses.
 
-## Add IP addresses to the cluster
+For example, a cluster with 180 worker nodes needs at least three IP addresses (180 nodes / 62 nodes per IP).
 
-A maximum of 20 IP addresses can be added to a load balancer. One (1) IP address is needed per 65 nodes, so a cluster with 250 nodes requires a minimum of four (4) IP addresses.
+This can be accomplished as part of the cluster creation process or later, after the cluster is created.
 
-To add IP addresses to the load balancer using Azure CLI, run the following command:
+## Deploy a cluster
+
+> [!NOTE]
+> When deploying a large cluster you must start with at most 50 worker nodes at creation time, then scale the cluster out to the desired number of worker nodes up to 250 worker nodes. 
+> 
+
+> [!NOTE]
+> While you can define up to 50 worker nodes at creation time, it is a best practice to start with a small cluster (e.g, 3 worker nodes as the default) and then scale to the desired number of worker nodes after the cluster is installed.
+>
 
-`az aro update -n [clustername] –g [resourcegroup]  --lb-ip-count 20`
+Follow the steps listed on the [Create an Azure Red Hat OpenShift cluster](https://learn.microsoft.com/azure/openshift/create-cluster?tabs=azure-cli) page until the "Create the cluster" steps, then continue as instructed:
 
-To add IP addresses through a rest API call:
+For large clusters, the previous recommendations should be followed. Therefore, the sample command below using the Azure CLI can be used to deploy a cluster with Standard_D32s_v5 as the control plane nodes, request three public IP addresses, and define nine worker nodes:
 
-```rest
-az rest --method patch --url https://management.azure.com/subscriptions/fe16a035-e540-4ab7-80d9-373fa9a3d6ae/resourceGroups/shared-cluster/providers/Microsoft.RedHatOpenShift/OpenShiftClusters/shared-cluster?api-version=2023-07-01-preview --body '{"properties": {"networkProfile": {"loadBalancerProfile": {"managedOutboundIps": {"count": 5}}}}}' --headers "Content-Type=application/json"
+```azurecli
+az aro create \ 
+    --resource-group $RESOURCEGROUP \
+    --name $CLUSTER \
+    --vnet aro-vnet \
+    --master-subnet master-subnet \
+    --worker-subnet worker-subnet \
+    --master-vm-size Standard_D32s_v5 \
+    --worker-count 9 \
+    --lb-ip-count 3
 ```
 
+To add IP addresses to the load balancer using the Azure CLI after the cluster is created, run the following command:
+
+`az aro update --name <CLUSTER_NAME> –-resource-group <RESOURCE_GROUP> --lb-ip-count <PUBLIC_IP_COUNT>`
+
+You can then configure the corresponding OpenShift MachineSets to obtain the number of worker nodes desired. See [Manually scaling a compute machine set](https://docs.openshift.com/container-platform/latest/machine_management/manually-scaling-machineset.html) for more details.
+
+After the cluster is successfully installed, proceed to deploying infrastructure nodes as defined in the [Infrastructure nodes](#infrastructure-nodes) section above.