Merge pull request #286618 from johnmarco/jm-aro-large-clusters-updates

ARO: Large cluster article changes

Author: v-shils

articles/openshift/howto-large-clusters.md

@@ -5,56 +5,78 @@ author: johnmarco
 ms.author: johnmarc
 ms.service: azure-redhat-openshift
 ms.topic: article
-ms.date: 08/15/2024
+ms.date: 09/12/2024
 ---
 # Deploy a large Azure Red Hat OpenShift cluster
 
-This article provides the steps and best practices for deploying large scale Azure Red Hat OpenShift clusters up to 250 nodes. For clusters of that size, a combination of control plane nodes and infrastructure nodes is needed to ensure the cluster functions properly is recommended.
+This article provides the steps and best practices for deploying large scale Azure Red Hat OpenShift clusters up to 250 worker nodes. For clusters of that size, there are some recommendations regarding control plane nodes and infrastructure nodes.
 
 > [!CAUTION]
-> Before deleting a large cluster, descale the cluster to 120 nodes or below.
+> Before deleting a cluster with more than 120 nodes, scale down the cluster to 120 nodes or less.
 > 
 
-## Deploy a cluster
+## Recommendations
+
+### Control plane nodes
 
-For clusters with over 101 control plane nodes, use the following [virtual machine instance types](support-policies-v4.md#supported-virtual-machine-sizes) size recommendations (or similar, newer generation instance types):
+For clusters with over 100 worker nodes, the following [virtual machine instance types](support-policies-v4.md#supported-virtual-machine-sizes) (or similar, newer generation instance types) are recommended for control plane nodes:
 
 - Standard_D32s_v3
 - Standard_D32s_v4
 - Standard_D32s_v5
 
-Following is a sample script using Azure CLI to deploy a cluster with Standard_D32s_v5 as the control plane node:
-
-```azurecli
-#az aro create \ --resource-group $RESOURCEGROUP \ --name $CLUSTER \ --vnet aro-vnet \ --master-subnet master-subnet \ --worker-subnet worker-subnet --master-vm-size Standard_D32s_v5
-```
-
-## Deploy infrastructure nodes for the cluster
+### Infrastructure nodes
 
-For clusters with over 101 nodes, infrastructure nodes are required to separate cluster workloads (such as prometheus) to minimize contention with other workloads.
- 
-> [!NOTE]
-> It's recommended that you deploy three (3) infrastructure nodes per cluster for redundancy and scalability needs. 
-> 
+For clusters with over 100 worker nodes, infrastructure nodes are required to separate cluster workloads (such as Prometheus) to minimize contention with other workloads. You should deploy three (3) infrastructure nodes per cluster for redundancy and scalability needs.
 
 The following instance types are recommended for infrastructure nodes:
 
 - Standard_E16as_v5
 - Standard_E16s_v5
 
-For instructions on configuring infrastructure nodes, see [Deploy infrastructure nodes in an Azure Red Hat OpenShift cluster](howto-infrastructure-nodes.md).
+For instructions on configuring infrastructure nodes, see [Deploy infrastructure nodes in an Azure Red Hat OpenShift cluster](howto-infrastructure-nodes.md). This will be configured after cluster deployment.
+
+### Add IP addresses to the load balancer
+
+Azure Red Hat OpenShift public clusters are created with a public load balancer that's used for outbound connectivity from inside the cluster. By default, one public IP address is configured on that public load balancer, and that limits the maximum node count of your cluster to 62. To be able to scale your cluster to the maximum supported number of 250 nodes, you need to assign multiple additional public IP addresses to the load balancer. You can configure up to 20 IP addresses per cluster. The outbound rules and frontend IP configurations are adjusted to accommodate the number of IP addresses.
 
-## Add IP addresses to the cluster
+For example, a cluster with 180 worker nodes needs at least (3) three IP addresses (180 nodes / 62 nodes per IP).
 
-A maximum of 20 IP addresses can be added to a load balancer. One (1) IP address is needed per 65 nodes, so a cluster with 250 nodes requires a minimum of four (4) IP addresses.
+This can be accomplished as part of the cluster creation process or later, after the cluster is created.
 
-To add IP addresses to the load balancer using Azure CLI, run the following command:
+## Deploy a cluster
+
+When deploying a large cluster, you must start with at most 50 worker nodes at creation time, then scale the cluster out to the desired number of worker nodes, up to 250 worker nodes. 
+
+> [!NOTE]
+> While you can define up to 50 worker nodes at creation time, it's best to start with a small cluster (e.g, three (3) worker nodes) and then scale out to the desired number of worker nodes after the cluster is installed.
+>
+
+Follow the steps provided in [Create an Azure Red Hat OpenShift cluster](https://learn.microsoft.com/azure/openshift/create-cluster?tabs=azure-cli) until the "Create the cluster" steps, then continue as instructed:
 
-`az aro update -n [clustername] –g [resourcegroup]  --lb-ip-count 20`
+The sample command below using the Azure CLI can be used to deploy a cluster with Standard_D32s_v5 as the control plane nodes, requesting three public IP addresses, and defining nine worker nodes:
 
-To add IP addresses through a rest API call:
+```azurecli
+az aro create \ 
+    --resource-group $RESOURCEGROUP \
+    --name $CLUSTER \
+    --vnet aro-vnet \
+    --master-subnet master-subnet \
+    --worker-subnet worker-subnet \
+    --master-vm-size Standard_D32s_v5 \
+    --worker-count 9 \
+    --lb-ip-count 3
+```
+
+To add IP addresses to the load balancer using the Azure CLI after the cluster is created, run the following command:
 
-```rest
-az rest --method patch --url https://management.azure.com/subscriptions/fe16a035-e540-4ab7-80d9-373fa9a3d6ae/resourceGroups/shared-cluster/providers/Microsoft.RedHatOpenShift/OpenShiftClusters/shared-cluster?api-version=2023-07-01-preview --body '{"properties": {"networkProfile": {"loadBalancerProfile": {"managedOutboundIps": {"count": 5}}}}}' --headers "Content-Type=application/json"
+```azurecli
+az aro update
+    --name <CLUSTER_NAME>
+    –-resource-group <RESOURCE_GROUP>
+    --lb-ip-count <PUBLIC_IP_COUNT>`
 ```
 
+You can then configure the corresponding OpenShift MachineSets to obtain the number of worker nodes desired. See [Manually scaling a compute machine set](https://docs.openshift.com/container-platform/latest/machine_management/manually-scaling-machineset.html) for more details.
+
+Once the cluster is successfully installed, proceed to deploying infrastructure nodes as defined in the [Infrastructure nodes](#infrastructure-nodes) section.