Merge pull request #263298 from dennispadia/dp-hanalbupdate

Update load balancer steps

Author: JamesJBarnett

articles/sap/workloads/disaster-recovery-sap-hana.md

@@ -7,28 +7,30 @@ ms.service: sap-on-azure
 ms.subservice: sap-vm-workloads
 ms.workload: infrastructure-services
 ms.topic: how-to
-ms.date: 12/08/2023
+ms.date: 01/16/2024
 ms.custom: template-how-to-pattern 
 ---
 
 # Add HSR third site to HANA Pacemaker cluster
 
 This article describes requirements and setup of a third HANA replication site to complement an existing Pacemaker cluster. Both SUSE Linux Enterprise Server (SLES) and RedHat Enterprise Linux (RHEL) specifics are covered.
 
-## Overview 
+## Overview
 
 SAP HANA supports system replication (HSR) with more than two sites connected. You can add a third site to an existing HSR pair, managed by Pacemaker in a highly available setup. You can deploy the third site in a second Azure region for disaster recovery (DR) purposes.  
 
 Pacemaker and HANA cluster resource agent manage the first two sites. Pacemaker cluster doesn't control the third site.
 
-SAP HANA supports a third system replication site in two modes. 
+SAP HANA supports a third system replication site in two modes.
+
 - [Multi-target](https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/ba457510958241889a459e606bbcf3d3.html) replicates data changes from primary to more than one target system. Third site connected to primary, replication in a star topology.
 - [Multi-tier](https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/f730f308fede4040bcb5ccea6751e74d.html) is a two-tier replication. A cascading, or sometimes referred to as chained setup, of three different HANA tiers. Third site connects to secondary.  
+
 For more information, see [SAP HANA availability across Azure regions](./sap-hana-availability-across-regions.md#combine-availability-within-one-region-and-across-regions) for more conceptual details about HANA HSR within one and across different regions.
 
 ## Prerequisites for SLES
 
-Requirements for a third HSR site are different between HANA scale-up and HANA scale-out. 
+Requirements for a third HSR site are different between HANA scale-up and HANA scale-out.
 
 > [!NOTE]
 > Requirements in this chapter are only valid for a Pacemaker enabled landscape. Without Pacemaker, SAP HANA version requirements apply for the chosen replication mode.  
@@ -42,7 +44,7 @@ Requirements for a third HSR site are different between HANA scale-up and HANA s
 
 ## Prerequisites for RHEL
 
-Requirements for a third HSR site are different between HANA scale-up and HANA scale-out. 
+Requirements for a third HSR site are different between HANA scale-up and HANA scale-out.
 
 > [!NOTE]
 > Requirements in this chapter are only valid for a Pacemaker enabled landscape. Without Pacemaker, SAP HANA version requirements apply for the chosen replication mode.  
@@ -60,18 +62,23 @@ Failure of the third node won't trigger any cluster action. Cluster detects the
 Example of a multi-target system replication system. For more information, see [SAP documentation](https://help.sap.com/docs/SAP_HANA_PLATFORM/4e9b18c116aa42fc84c7dbfd02111aba/2e6c71ab55f147e19b832565311a8e4e.html).  
 ![Diagram showing an example of a HANA scale-up multi-target system replication system.](./media/sap-hana-high-availability/sap-hana-high-availability-scale-up-hsr-multi-target.png)
 
-1. Deploy Azure resources for the third node. Depending on your requirements, you can use a different Azure region for disaster recovery purposes.  
-   Steps required for the third site are similar to [virtual machines for HANA scale-up cluster](./sap-hana-high-availability.md#deploy-for-linux). Third site will use Azure infrastructure, operating system and HANA version matching the existing Pacemaker cluster, with the following exceptions:
+1. Deploy Azure resources for the third node. Depending on your requirements, you can use a different Azure region for disaster recovery purposes.
+
+   Steps required for the third site are similar to [virtual machines for HANA scale-up cluster](./sap-hana-high-availability.md#prepare-the-infrastructure). Third site will use Azure infrastructure, operating system and HANA version matching the existing Pacemaker cluster, with the following exceptions:
+
    - No load balancer deployed for third site and no integration with existing cluster load balancer for the VM of third site
    - Don't install OS packages SAPHanaSR, SAPHanaSR-doc and OS package pattern ha_sles on third site VM
    - No integration into the cluster for VM or HANA resources of the third site
    - No HANA HA hook setup for third site in global.ini
-   
-2. Install SAP HANA on third node.  
+
+2. Install SAP HANA on third node.
+  
    Same HANA SID and HANA installation number must be used for third site.  
-    
-3. With SAP HANA on third site installed and running, register the third site with the primary site.  
+
+3. With SAP HANA on third site installed and running, register the third site with the primary site.
+
    The example uses SITE-DR as the name for third site.
+
     ```bash
     # Execute on the third site 
     su - hn1adm
@@ -80,12 +87,14 @@ Example of a multi-target system replication system. For more information, see [
     ```
 
 4. Verify HANA system replication shows both secondary and third site.
+
     ```bash
     # Verify HANA HSR is in sync, execute on primary
     sudo su - hn1adm -c "python /usr/sap/HN1/HDB03/exe/python_support/systemReplicationStatus.py"
     ```
 
 5. Check the SAPHanaSR attribute for third site. SITE-DR should show up with status SOK in the sites section.
+
     ```bash
     # Check SAPHanaSR attribute on any cluster managed host (first or second site)
     sudo SAPHanaSR-showAttr
@@ -100,7 +109,7 @@ Example of a multi-target system replication system. For more information, see [
     # HN1-SITE2 SOK
     # SITE-DR   SOK
     ```
-   
+
    Cluster detects the replication status of connected sites and the monitored attributed can change between SOK and SFAIL. No cluster action if the replication to DR site fails.
 
 ## HANA scale-out: Add HANA multi-target system replication for DR purposes
@@ -112,19 +121,23 @@ Failure of the third node won't trigger any cluster action. Cluster detects the
 Example of a multi-target system replication system. For more information, see [SAP documentation](https://help.sap.com/docs/SAP_HANA_PLATFORM/4e9b18c116aa42fc84c7dbfd02111aba/2e6c71ab55f147e19b832565311a8e4e.html).  
 ![Diagram showing an example of a HANA scale-out multi-target system replication system.](./media/sap-hana-high-availability/sap-hana-high-availability-scale-out-hsr-multi-target.png)
 
-1. Deploy Azure resources for the third site. Depending on your requirements, you can use a different Azure region for disaster recovery purposes.  
-   Steps required for the HANA scale-out on third site are mirroring steps to deploy the [HANA scale-out cluster](./sap-hana-high-availability-scale-out-hsr-suse.md#set-up-the-infrastructure). Third site will use Azure infrastructure, operating system and HANA installation steps for SITE1 of the scale-out cluster, with the following exceptions:
+1. Deploy Azure resources for the third site. Depending on your requirements, you can use a different Azure region for disaster recovery purposes.
+
+   Steps required for the HANA scale-out on third site are mirroring steps to deploy the [HANA scale-out cluster](./sap-hana-high-availability-scale-out-hsr-suse.md#prepare-the-infrastructure). Third site will use Azure infrastructure, operating system and HANA installation steps for SITE1 of the scale-out cluster, with the following exceptions:
+
    - No load balancer deployed for third site and no integration with existing cluster load balancer for the VMs of third site
    - Don't install OS packages SAPHanaSR-ScaleOut, SAPHanaSR-ScaleOut-doc and OS package pattern ha_sles on third site VMs
    - No majority maker VM for third site, as there's no cluster integration
    - Create NFS volume /hana/shared for third site exclusive use
    - No integration into the cluster for VMs or HANA resources of the third site
    - No HANA HA hook setup for third site in global.ini
-   
-   You must use the same HANA SID and HANA installation number for third site. 
-    
-2. With SAP HANA scale-out on third site installed and running, register the third site with the primary site.  
+
+   You must use the same HANA SID and HANA installation number for third site.
+
+2. With SAP HANA scale-out on third site installed and running, register the third site with the primary site.
+
    The example uses SITE-DR as the name for third site.
+
     ```bash
     # Execute on the third site 
     su - hn1adm
@@ -133,12 +146,14 @@ Example of a multi-target system replication system. For more information, see [
     ```
 
 3. Verify HANA system replication shows both secondary and third site.
+
     ```bash
     # Verify HANA HSR is in sync, execute on primary
     sudo su - hn1adm -c "python /usr/sap/HN1/HDB03/exe/python_support/systemReplicationStatus.py"
     ```
 
 4. Check the SAPHanaSR attribute for third site. SITE-DR should show up with status SOK in the sites section.
+
     ```bash
     # Check SAPHanaSR attribute on any cluster managed host (first or second site)
     sudo SAPHanaSR-showAttr
@@ -153,7 +168,7 @@ Example of a multi-target system replication system. For more information, see [
     # HANA_S1   1674815869 4   hana-s1-db1 PRIM   P
     # HANA_S2   30         4   hana-s2-db1 SOK    S
     ```
-   
+
    Cluster detects the replication status of connected sites and the monitored attributed can change between SOK and SFAIL. No cluster action if the replication to DR site fails.
 
 ## Autoregistering third site
@@ -164,10 +179,10 @@ SAP provides since HANA 2 SPS 04 parameter `register_secondaries_on_takeover`. W
 
 For HSR [multi-tier](https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/f730f308fede4040bcb5ccea6751e74d.html), no automatic SAP HANA registration of the third site exists. You need to manually register the third site to the current secondary, to keep HSR replication chain for multi-tier.
 
-![Diagram flow showing how a HANA auto-registartion works with a third site during a takeover.](./media/sap-hana-high-availability/sap-hana-high-availability-hsr-third-site-auto-register.png)
+![Diagram flow showing how a HANA auto-registration works with a third site during a takeover.](./media/sap-hana-high-availability/sap-hana-high-availability-hsr-third-site-auto-register.png)
 
 ## Next steps
 
 - [Disaster recovery overview and infrastructure](./disaster-recovery-overview-guide.md)
 - [Disaster recovery for SAP workloads](./disaster-recovery-sap-guide.md)
-- [High-availability architecture and scenarios for SAP NetWeaver](./sap-hana-availability-across-regions.md)
+- [High-availability architecture and scenarios for SAP NetWeaver](./sap-hana-availability-across-regions.md)

articles/sap/workloads/sap-hana-high-availability-netapp-files-suse.md

@@ -11,7 +11,7 @@ ms.service: sap-on-azure
 ms.subservice: sap-vm-workloads
 ms.topic: tutorial
 ms.workload: infrastructure-services
-ms.date: 09/15/2023
+ms.date: 01/16/2024
 ms.author: ampatel
 ---
 
@@ -60,7 +60,7 @@ Read the following SAP Notes and papers first:
 - [NFS v4.1 volumes on Azure NetApp Files for SAP HANA](./hana-vm-operations-netapp.md)
 - [Azure Virtual Machines planning and implementation for SAP on Linux](./planning-guide.md)
 
->[!NOTE]
+> [!NOTE]
 > This article contains references to a term that Microsoft no longer uses. When the term is removed from the software, we’ll remove it from this article.
 
 ## Overview
@@ -108,12 +108,12 @@ As you create your Azure NetApp Files for SAP HANA Scale-up systems, be aware of
 
 The throughput of an Azure NetApp Files volume is a function of the volume size and service level, as documented in [Service level for Azure NetApp Files](../../azure-netapp-files/azure-netapp-files-service-levels.md).
 
-While designing the infrastructure for SAP HANA on Azure with Azure NetApp Files, be aware of the recommendations in [NFS v4.1 volumes on Azure NetApp Files for SAP HANA](./hana-vm-operations-netapp.md#sizing-for-hana-database-on-azure-netapp-files).   
+While designing the infrastructure for SAP HANA on Azure with Azure NetApp Files, be aware of the recommendations in [NFS v4.1 volumes on Azure NetApp Files for SAP HANA](./hana-vm-operations-netapp.md#sizing-for-hana-database-on-azure-netapp-files).
 
-The configuration in this article is presented with simple Azure NetApp Files Volumes. 
+The configuration in this article is presented with simple Azure NetApp Files Volumes.
 
 > [!IMPORTANT]
-> For production systems, where performance is a key, we recommend to evaluate and consider using [Azure NetApp Files application volume group for SAP HANA](hana-vm-operations-netapp.md#deployment-through-azure-netapp-files-application-volume-group-for-sap-hana-avg).   
+> For production systems, where performance is a key, we recommend to evaluate and consider using [Azure NetApp Files application volume group for SAP HANA](hana-vm-operations-netapp.md#deployment-through-azure-netapp-files-application-volume-group-for-sap-hana-avg).
 
 > [!NOTE]
 > All commands to mount /hana/shared in this article are presented for NFSv4.1 /hana/shared volumes.
@@ -144,46 +144,36 @@ The following instructions assume that you've already deployed your [Azure virtu
    - Volume hanadb2-log-mnt00001 (nfs://10.3.1.4:/hanadb2-log-mnt00001)
    - Volume hanadb2-shared-mnt00001 (nfs://10.3.1.4:/hanadb2-shared-mnt00001)
 
+## Prepare the infrastructure
+
+The resource agent for SAP HANA is included in SUSE Linux Enterprise Server for SAP Applications. An image for SUSE Linux Enterprise Server for SAP Applications 12 or 15 is available in Azure Marketplace. You can use the image to deploy new VMs.
 
-## Deploy Linux virtual machine via Azure portal
+### Deploy Linux VMs manually via Azure portal
 
 This document assumes that you've already deployed a resource group, [Azure Virtual Network](../../virtual-network/virtual-networks-overview.md), and subnet.
 
-Deploy virtual machines for SAP HANA. Choose a suitable SLES image that is supported for HANA system. You can deploy VM in any one of the availability options - scale set, availability zone or availability set.
+Deploy virtual machines for SAP HANA. Choose a suitable SLES image that is supported for HANA system. You can deploy VM in any one of the availability options - virtual machine scale set, availability zone or availability set.
 
 > [!IMPORTANT]
 > Make sure that the OS you select is SAP certified for SAP HANA on the specific VM types that you plan to use in your deployment. You can look up SAP HANA-certified VM types and their OS releases in [SAP HANA Certified IaaS Platforms](https://www.sap.com/dmc/exp/2014-09-02-hana-hardware/enEN/#/solutions?filters=v:deCertified;ve:24;iaas;v:125;v:105;v:99;v:120). Make sure that you look at the details of the VM type to get the complete list of SAP HANA-supported OS releases for the specific VM type.
 
-During VM configuration, we won't be adding any disk as all our mount points are on NFS shares from Azure NetApp Files. Also, you have an option to create or select exiting load balancer in networking section. If you're creating a new load balancer, follow below steps -
-
-1. To set up standard load balancer, follow these configuration steps:
-   1. First, create a front-end IP pool:
-      1. Open the load balancer, select **frontend IP configuration**, and select **Add**.
-      2. Enter the name of the new front-end IP (for example, **hana-frontend**).
-      3. Set the **Assignment** to **Static** and enter the IP address (for example, **10.3.0.50**).
-      4. Select **OK**.
-      5. After the new front-end IP pool is created, note the pool IP address.
-   2. Create a single back-end pool:
-      1. Open the load balancer, select **Backend pools**, and then select **Add**.
-      2. Enter the name of the new back-end pool (for example, **hana-backend**).
-      3. Select **NIC** for Backend Pool Configuration.
-      4. Select **Add a virtual machine**.
-      5. Select the virtual machines of the HANA cluster.
-      6. Select **Add**.
-      7. Select **Save**.
-   3. Next, create a health probe:
-      1. Open the load balancer, select **health probes**, and select **Add**.
-      2. Enter the name of the new health probe (for example, **hana-hp**).
-      3. Select TCP as the protocol and port 625**03**. Keep the **Interval** value set to 5.
-      4. Select **OK**.
-   4. Next, create the load-balancing rules:
-      1. Open the load balancer, select **load balancing rules**, and select **Add**.
-      2. Enter the name of the new load balancer rule (for example, **hana-lb**).
-      3. Select the front-end IP address, the back-end pool, and the health probe that you created earlier (for example, **hana-frontend**, **hana-backend** and **hana-hp**).
-         1. Increase idle timeout to 30 minutes
-      4. Select **HA Ports**.
-      5. Make sure to **enable Floating IP**.
-      6. Select **OK**.
+### Configure Azure load balancer
+
+During VM configuration, you have an option to create or select exiting load balancer in networking section. Follow below steps, to setup standard load balancer for high availability setup of HANA database.
+
+#### [Azure Portal](#tab/lb-portal)
+
+[!INCLUDE [Configure Azure standard load balancer using Azure portal](../../../includes/sap-load-balancer-db-portal.md)]
+
+#### [Azure CLI](#tab/lb-azurecli)
+
+[!INCLUDE [Configure Azure standard load balancer using Azure CLI](../../../includes/sap-load-balancer-db-azurecli.md)]
+
+#### [PowerShell](#tab/lb-powershell)
+
+[!INCLUDE [Configure Azure standard load balancer using PowerShell](../../../includes/sap-load-balancer-db-powershell.md)]
+
+---
 
 For more information about the required ports for SAP HANA, read the chapter [Connections to Tenant Databases](https://help.sap.com/viewer/78209c1d3a9b41cd8624338e42a12bf6/latest/en-US/7a9343c9f2a2436faa3cfdb5ca00c052.html) in the [SAP HANA Tenant Databases](https://help.sap.com/viewer/78209c1d3a9b41cd8624338e42a12bf6) guide or SAP Note [2388694](https://launchpad.support.sap.com/#/notes/2388694).
 
@@ -194,7 +184,9 @@ For more information about the required ports for SAP HANA, read the chapter [Co
 > When VMs without public IP addresses are placed in the backend pool of internal (no public IP address) Standard Azure load balancer, there will be no outbound internet connectivity, unless additional configuration is performed to allow routing to public end points. For details on how to achieve outbound connectivity see [Public endpoint connectivity for Virtual Machines using Azure Standard Load Balancer in SAP high-availability scenarios](./high-availability-guide-standard-load-balancer-outbound-connections.md).
 
 > [!IMPORTANT]
-> Do not enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps will cause the health probes to fail. Set parameter **net.ipv4.tcp_timestamps** to **0**. For details see [Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md). See also SAP note [2382421](https://launchpad.support.sap.com/#/notes/2382421).
+>
+> - Do not enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps will cause the health probes to fail. Set parameter `net.ipv4.tcp_timestamps` to `0`. For details see [Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md) and SAP note [2382421](https://launchpad.support.sap.com/#/notes/2382421).
+> - To prevent saptune from changing the manually set `net.ipv4.tcp_timestamps` value from `0` back to `1`, you should update saptune version to 3.1.1 or higher. For more details, see [saptune 3.1.1 – Do I Need to Update?](https://www.suse.com/c/saptune-3-1-1-do-i-need-to-update/).
 
 ## Mount the Azure NetApp Files volume