Merge pull request #260737 from johnmarco/jm-aro-nsg-enhancement

ARO NSG enhancement article

Author: v-dirichards

articles/openshift/howto-bring-nsg.md

@@ -0,0 +1,105 @@
+---
+title: Bring your own Network Security Group to Azure Red Hat OpenShift
+description: In this article, learn how to bring your own Network Security Group (NSG) to an Azure Red Hat OpenShift cluster.
+author: johnmarco
+ms.service: azure-redhat-openshift
+ms.topic: how-to
+ms.author: johnmarc
+ms.date: 05/06/2024
+topic: how-to
+recommendations: true
+keywords: azure, openshift, aro, NSG
+#Customer intent: I need to attach my own Network Security Group to an ARO cluster before beginning cluster installation.
+---
+
+# Bring your own Network Security Group (NSG) to an Azure Red Hat OpenShift (ARO) cluster
+
+Typically, when setting up an ARO cluster, you must designate a resource group for deploying the ARO cluster object (referred to as the Base Resource Group in the following diagram). In such scenarios, you can use either the same resource group for both the virtual network (VNET) and the cluster, or you can opt for a separate resource group solely for the VNET. Neither of these resource groups directly corresponds to a single ARO cluster, granting you complete control over them. This means you can freely create, modify, or delete resources within these resource groups.
+
+During the cluster creation process, the ARO Resource Provider (RP) establishes a dedicated resource group specific to the cluster's needs. This group houses various cluster-specific resources like node VMs, load balancers, and Network Security Groups (NSGs), as depicted by the Managed Resource Group in the following diagram. The Managed Resource Group is tightly secured, prohibiting any modifications to its contents, including the NSG linked to the VNET subnets specified during cluster creation. In some situations, the NSG generated by the ARO RP might not adhere to the security policies of certain organizations.
+
+:::image type="content" source="media/howto-bring-nsg/network-security-group-old.png" alt-text="Diagram showing an overview of how network security groups work in a typical ARO cluster." lightbox="media/howto-bring-nsg/network-security-group-old.png":::
+
+This article shows how to use the "bring your own" Network Security Group (NSG) feature to attach your own preconfigured NSG residing in the Base/VNET resource group (RG) (shown in the following diagram as BYO-NSG) to the ARO cluster subnets. Since you own this preconfigured NSG, you can add/remove rules during the lifetime of the ARO cluster.
+
+:::image type="content" source="media/howto-bring-nsg/network-security-group-new.png" alt-text="Diagram showing an overview of how to bring your own network security group works in Azure Red Hat OpenShift." lightbox="media/howto-bring-nsg/network-security-group-new.png":::
+
+## General capabilities and limitations
+
+- You need to attach your preconfigured NSGs to both master and worker subnets before you create the cluster. Failure to attach your preconfigured NSGs to both subnets results in an error. 
+
+- You can choose to use the same or different preconfigured NSGs for master and worker subnets.
+
+- When using your own NSG, the ARO RP still creates an NSG in the Managed Resource Group (default NSG), but that NSG isn't attached to the worker or master subnets.
+
+- You can't enable the preconfigured NSG feature on an existing ARO cluster. Currently, this feature can only be enabled at the time of cluster creation.
+
+- The preconfigured NSG option isn't configurable from the Azure portal.
+
+- If you used this feature during preview, your existing preconfigured clusters are now fully supported. 
+
+### Using rules
+
+> [!WARNING]
+> Preconfigured NSGs aren't automatically updated with rules when you create Kubernetes LoadBalancer type services or OpenShift routes within the ARO cluster. Therefore, you must update these rules manually, as required. This behavior is different from the original ARO behavior wherein the default NSG is programmatically updated in such situations.
+> 
+
+- The default ARO cluster NSG (not attached to any subnet while using this feature) will still be updated with rules when you create Kubernetes LoadBalancer type services or OpenShift routes within the ARO cluster.
+
+- You can detach preconfigured NSGs from the subnets of the cluster created using this feature. It results in a cluster with subnets that have no NSGs. You can then attach a different set of preconfigured NSGs to the cluster. Alternatively, you can attach the ARO default NSG to the cluster subnets (at which point your cluster becomes like any other cluster that's not using this feature).
+
+- Your preconfigured NSGs shouldn't have INBOUND/OUTBOUND DENY rules of the following types, as these can interfere with the operation of the cluster and/or hinder the ARO support/SRE teams from providing support/management. (Here, subnet indicates any or all IP addresses in the subnet and all ports corresponding to that subnet):
+
+    - Master Subnet ←→ Master Subnet
+    - Worker Subnet ←→ Worker Subnet
+    - Master Subnet ←→ Worker Subnet
+    
+    - Misconfigured rules result in a [signal used by Azure Monitor](/azure/openshift/howto-monitor-alerts) to help troubleshoot preconfigured NSGs.
+       
+- To allow incoming traffic to your ARO public cluster, set the following INBOUND ALLOW rules (or equivalent) in your NSG. Refer to the default NSG of the cluster for specific details and to the example NSG shown in [Deployment](#deployment). You can create a cluster even without such rules in the NSG.
+
+    - For API server access → From Internet (or your preferred source IPs) to port 6443 on the master subnet.
+    - For access to OpenShift router (and hence to OpenShift console and OpenShift routes) → From Internet (or your preferred source IPs) to ports 80 and 443 on the default-v4 public IP on the public Load-balancer of the cluster.
+    - For access to any Load-balancer type Kubernetes service → From Internet (or your preferred source IPs) to service ports on public IP corresponding to the service on the public Load-balancer of the cluster.
+
+## Deployment
+
+### Create VNET and create and configure preconfigured NSG
+
+1. Create a VNET, and then create master and worker subnets within it.
+
+1. Create preconfigured NSGs with default rules (or no rules at all) and attach them to the master and worker subnets.
+
+### Create an ARO cluster and update preconfigured NSGs
+
+1. Create the cluster.
+
+    ```
+    az aro create \
+    --resource-group BASE_RESOURCE_GROUP_NAME \
+    --name CLSUTER_NAME \
+    --vnet VNET_NAME \
+    --master-subnet MASTER_SUBNET_NAME \
+    --worker-subnet WORKER_SUBNET_NAME \
+    --client-id CLUSTER_SERVICE_PRINCIPAL_ID \
+    --client-secret CLUSTER_SERVICE_PRINCIPAL_SECRET \
+    --enable-preconfigured-nsg
+    ```
+    
+1. Update the preconfigured NSGs with rules as per your requirements while also considering the points mentioned in [Capabilities and limitations](#general-capabilities-and-limitations).
+
+    The following example has the Cluster Public Load-balancer as shown in the screenshot/CLI output:
+    
+    :::image type="content" source="media/howto-bring-nsg/ip-configuration-load-balancer.png" alt-text="Screenshot of the cluster's public load balancer as shown with the output from the command." lightbox="media/howto-bring-nsg/ip-configuration-load-balancer.png":::
+   
+    ```Output
+    $ oc get svc | grep tools
+    tools LoadBalancer 172.30.182.7 20.141.176.3 80:30520/TCP 143m
+    $ $ oc get svc -n openshift-ingress | grep Load
+    router-default LoadBalancer 172.30.105.218 20.159.139.208 80:31157/TCP,443:31177/TCP 
+    5d20
+    ```
+    
+    :::image type="content" source="media/howto-bring-nsg/load-balancer-output.png" alt-text="Screenshot showing inbound and outbound security rules." lightbox="media/howto-bring-nsg/load-balancer-output.png":::
+
+

articles/openshift/howto-enable-nsg-flowlogs.md

@@ -1,7 +1,7 @@
 ---
 title: Enabling Network Security Group flow logs for Azure Red Hat OpenShift
 description: In this article, learn how to enable flow logs to analyze traffic for Network Security Groups.
-author: johnmarc
+author: johnmarco
 ms.service: azure-redhat-openshift
 ms.topic: how-to
 ms.author: johnmarc