Changes in xNFS and SLES pacemaker documents

Author: dennispadia

articles/sap/workloads/high-availability-guide-rhel-nfs-azure-files.md

@@ -9,7 +9,7 @@ ms.service: sap-on-azure
 ms.subservice: sap-vm-workloads
 ms.topic: tutorial
 ms.workload: infrastructure-services
-ms.date: 01/18/2024
+ms.date: 02/05/2024
 ms.author: radeltch
 ---
 
@@ -251,7 +251,7 @@ The following items are prefixed with:
     ```bash
     # mount temporarily the volume
     sudo mkdir -p /saptmp
-    sudo mount -t nfs sapnfs.file.core.windows.net:/sapnfsafs/sapnw1 /saptmp -o vers=4,minorversion=1,sec=sys
+    sudo mount -t nfs sapnfs.file.core.windows.net:/sapnfsafs/sapnw1 /saptmp -o noresvport,vers=4,minorversion=1,sec=sys
     # create the SAP directories
     sudo cd /saptmp
     sudo mkdir -p sapmntNW1
@@ -293,9 +293,9 @@ The following items are prefixed with:
     ```bash
     vi /etc/fstab
     # Add the following lines to fstab, save and exit
-    sapnfs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs vers=4,minorversion=1,sec=sys  0  0
-    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs vers=4,minorversion=1,sec=sys  0  0
-    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1sys/ /usr/sap/NW1/SYS  nfs vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1sys/ /usr/sap/NW1/SYS  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
     
     # Mount the file systems
     mount -a 
@@ -345,7 +345,7 @@ The following items are prefixed with:
     sudo pcs node standby sap-cl2
    
     sudo pcs resource create fs_NW1_ASCS Filesystem device='sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1ascs' \
-      directory='/usr/sap/NW1/ASCS00' fstype='nfs' force_unmount=safe options='sec=sys,vers=4.1' \
+      directory='/usr/sap/NW1/ASCS00' fstype='nfs' force_unmount=safe options='noresvport,vers=4,minorversion=1,sec=sys' \
       op start interval=0 timeout=60 op stop interval=0 timeout=120 op monitor interval=200 timeout=40 \
       --group g-NW1_ASCS
    
@@ -401,7 +401,7 @@ The following items are prefixed with:
     sudo pcs node standby sap-cl1
     
     sudo pcs resource create fs_NW1_AERS Filesystem device='sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1ers' \
-      directory='/usr/sap/NW1/ERS01' fstype='nfs' force_unmount=safe options='sec=sys,vers=4.1' \
+      directory='/usr/sap/NW1/ERS01' fstype='nfs' force_unmount=safe options='noresvport,vers=4,minorversion=1,sec=sys' \
       op start interval=0 timeout=60 op stop interval=0 timeout=120 op monitor interval=200 timeout=40 \
      --group g-NW1_AERS
    
@@ -687,8 +687,8 @@ The following items are prefixed with:
     ```bash
     vi /etc/fstab
     # Add the following lines to fstab, save and exit
-    sapnfs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs vers=4,minorversion=1,sec=sys  0  0
-    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
     
     # Mount the file systems
     mount -a 

articles/sap/workloads/high-availability-guide-suse-nfs-azure-files.md

@@ -9,7 +9,7 @@ ms.subservice: sap-vm-workloads
 ms.topic: tutorial
 ms.workload: infrastructure-services
 ms.custom: devx-track-azurecli, devx-track-azurepowershell
-ms.date: 01/17/2024
+ms.date: 02/05/2024
 ms.author: radeltch
 ---
 
@@ -267,7 +267,7 @@ The following items are prefixed with either **[A]** - applicable to all nodes,
     ```bash
     # mount temporarily the volume
     sudo mkdir -p /saptmp
-    sudo mount -t nfs sapnfs.file.core.windows.net:/sapnfsafs/sapnw1 /saptmp -o vers=4,minorversion=1,sec=sys
+    sudo mount -t nfs sapnfs.file.core.windows.net:/sapnfsafs/sapnw1 /saptmp -o noresvport,vers=4,minorversion=1,sec=sys
     # create the SAP directories
     sudo cd /saptmp
     sudo mkdir -p sapmntNW1
@@ -303,9 +303,9 @@ The following items are prefixed with either **[A]** - applicable to all nodes,
     ```bash
     vi /etc/fstab
     # Add the following lines to fstab, save and exit
-    sapnfs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs vers=4,minorversion=1,sec=sys  0  0
-    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs vers=4,minorversion=1,sec=sys  0  0
-    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1sys/ /usr/sap/NW1/SYS  nfs vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1sys/ /usr/sap/NW1/SYS  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
     
     # Mount the file systems
     mount -a 
@@ -347,7 +347,7 @@ The following items are prefixed with either **[A]** - applicable to all nodes,
 
     ```bash
     sudo crm node standby sap-cl2
-    sudo crm configure primitive fs_NW1_ASCS Filesystem device='sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1ascs' directory='/usr/sap/NW1/ASCS00' fstype='nfs' options='sec=sys,vers=4.1' \
+    sudo crm configure primitive fs_NW1_ASCS Filesystem device='sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1ascs' directory='/usr/sap/NW1/ASCS00' fstype='nfs' options='noresvport,vers=4,minorversion=1,sec=sys' \
       op start timeout=60s interval=0 \
       op stop timeout=60s interval=0 \
       op monitor interval=20s timeout=40s
@@ -402,7 +402,7 @@ The following items are prefixed with either **[A]** - applicable to all nodes,
     ```bash
     sudo crm node online sap-cl2
     sudo crm node standby sap-cl1
-    sudo crm configure primitive fs_NW1_ERS Filesystem device='sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1ers' directory='/usr/sap/NW1/ERS01' fstype='nfs' options='sec=sys,vers=4.1' \
+    sudo crm configure primitive fs_NW1_ERS Filesystem device='sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1ers' directory='/usr/sap/NW1/ERS01' fstype='nfs' options='noresvport,vers=4,minorversion=1,sec=sys' \
       op start timeout=60s interval=0 \
       op stop timeout=60s interval=0 \
       op monitor interval=20s timeout=40s
@@ -669,8 +669,8 @@ The following items are prefixed with either **[A]** - applicable to both PAS an
     ```bash
     vi /etc/fstab
     # Add the following lines to fstab, save and exit
-    sapnfs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs vers=4,minorversion=1,sec=sys  0  0
-    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
+    sapnfs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0
     
     # Mount the file systems
     mount -a 

articles/sap/workloads/high-availability-guide-suse-nfs-simple-mount.md

@@ -9,7 +9,7 @@ ms.subservice: sap-vm-workloads
 ms.topic: tutorial
 ms.workload: infrastructure-services
 ms.custom: devx-track-azurecli, devx-track-azurepowershell
-ms.date: 01/17/2024
+ms.date: 02/05/2024
 ms.author: radeltch
 ---
 
@@ -346,7 +346,7 @@ The following items are prefixed with:
     ```bash
     # Temporarily mount the volume.
     sudo mkdir -p /saptmp
-    sudo mount -t nfs sapnfsafs.file.core.windows.net:/sapnfsafs/sapnw1 /saptmp -o vers=4.1,sec=sys
+    sudo mount -t nfs sapnfsafs.file.core.windows.net:/sapnfsafs/sapnw1 /saptmp -o noresvport,vers=4,minorversion=1,sec=sys
     # Create the SAP directories.
     sudo cd /saptmp
     sudo mkdir -p sapmntNW1
@@ -374,9 +374,9 @@ The following items are prefixed with:
    With the simple mount configuration, the Pacemaker cluster doesn't control the file systems.
 
     ```bash
-    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1 nfs vers=4.1,sec=sys  0  0" >> /etc/fstab
-    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1/ /usr/sap/NW1 nfs vers=4.1,sec=sys  0  0" >> /etc/fstab
-    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans nfs vers=4.1,sec=sys  0  0" >> /etc/fstab   
+    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1 nfs noresvport,vers=4,minorversion=1,sec=sys  0  0" >> /etc/fstab
+    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/sapnw1/usrsapNW1/ /usr/sap/NW1 nfs noresvport,vers=4,minorversion=1,sec=sys  0  0" >> /etc/fstab
+    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans nfs noresvport,vers=4,minorversion=1,sec=sys  0  0" >> /etc/fstab   
     # Mount the file systems.
     mount -a 
     ```
@@ -843,8 +843,8 @@ If you're using NFS on Azure Files, use the following instructions to prepare th
 2. Mount the file systems.
 
     ```bash
-    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs vers=4.1,sec=sys  0  0" >> /etc/fstab
-    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs vers=4.1,sec=sys  0  0" >> /etc/fstab   
+    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/sapnw1/sapmntNW1 /sapmnt/NW1  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0" >> /etc/fstab
+    echo "sapnfsafs.file.core.windows.net:/sapnfsafs/saptrans /usr/sap/trans  nfs noresvport,vers=4,minorversion=1,sec=sys  0  0" >> /etc/fstab   
     # Mount the file systems.
     mount -a 
     ```

articles/sap/workloads/high-availability-guide-suse-pacemaker.md

@@ -384,6 +384,9 @@ Run the following commands on the nodes of the new cluster that you want to crea
     [...]
     ```
 
+    > [!NOTE]
+    > If the SBD_DELAY_START property value is set to "no", change the value to "yes". You must also check the SBD service file to ensure that the value of TimeoutStartSec is greater than the value of SBD_DELAY_START. For more information, see [SBD file configuraton](https://documentation.suse.com/sle-ha/15-SP5/html/SLE-HA-all/cha-ha-storage-protect.html#pro-ha-storage-protect-sbd-config)
+
 1. **[A]** Create the `softdog` configuration file.
 
     ```bash
@@ -409,39 +412,39 @@ This section applies only if you want to use an SBD device with an Azure shared
    $Location = "MyAzureRegion"
    ```
 
-1. Define the size of the disk based on available disk size for Premium SSDs. In this example, P1 disk size of 4G is mentioned.
+2. Define the size of the disk based on available disk size for Premium SSDs. In this example, P1 disk size of 4G is mentioned.
 
    ```bash
    $DiskSizeInGB = 4
    $DiskName = "SBD-disk1"
    ```
 
-1. With parameter -MaxSharesCount, define the maximum number of cluster nodes to attach the shared disk for the SBD device.
+3. With parameter -MaxSharesCount, define the maximum number of cluster nodes to attach the shared disk for the SBD device.
 
    ```bash
    $ShareNodes = 2
    ```
 
-1. For an SBD device that uses LRS for an Azure premium shared disk, use the following storage SkuName:
+4. For an SBD device that uses LRS for an Azure premium shared disk, use the following storage SkuName:
 
    ```bash
    $SkuName = "Premium_LRS"
    ```
 
-1. For an SBD device that uses ZRS for an Azure premium shared disk, use the following storage SkuName:
+5. For an SBD device that uses ZRS for an Azure premium shared disk, use the following storage SkuName:
 
    ```bash
    $SkuName = "Premium_ZRS"
    ```
 
-1. Set up an Azure shared disk.
+6. Set up an Azure shared disk.
 
    ```bash
    $diskConfig = New-AzDiskConfig -Location $Location -SkuName $SkuName -CreateOption Empty -DiskSizeGB $DiskSizeInGB -MaxSharesCount $ShareNodes
    $dataDisk = New-AzDisk -ResourceGroupName $ResourceGroup -DiskName $DiskName -Disk $diskConfig
    ```
 
-1. Attach the disk to the cluster VMs.
+7. Attach the disk to the cluster VMs.
 
    ```bash
    $VM1 = "prod-cl1-0"
@@ -468,7 +471,21 @@ If you want to deploy resources by using the Azure CLI or the Azure portal, you
 
 ### Set up an Azure shared disk SBD device
 
-1. **[A]** Make sure that the attached disk is available.
+1. **[A]** Install iSCSI package.
+
+   ```bash
+   sudo zypper install open-iscsi
+   ```
+
+2. **[A]** Enable the iSCSI and SBD services.
+
+   ```bash
+   sudo systemctl enable iscsid
+   sudo systemctl enable iscsi
+   sudo systemctl enable sbd
+   ```
+
+3. **[A]** Make sure that the attached disk is available.
 
    ```bash
    # lsblk
@@ -491,7 +508,7 @@ If you want to deploy resources by using the Azure CLI or the Azure portal, you
    [5:0:0:0]    disk    Msft     Virtual Disk     1.0   /dev/sdc
    ```
 
-1. **[A]** Retrieve the IDs of the attached disks.
+4. **[A]** Retrieve the IDs of the attached disks.
 
    ```bash
    # ls -l /dev/disk/by-id/scsi-* | grep sdc
@@ -501,15 +518,15 @@ If you want to deploy resources by using the Azure CLI or the Azure portal, you
 
    The commands list device IDs for the SBD device. We recommend using the ID that starts with scsi-3. In the preceding example, the ID is **/dev/disk/by-id/scsi-3600224804208a67da8073b2a9728af19**.
 
-1. **[1]** Create the SBD device.
+5. **[1]** Create the SBD device.
 
    Use the device ID from step 2 to create the new SBD devices on the first cluster node.
 
    ```bash
    # sudo sbd -d /dev/disk/by-id/scsi-3600224804208a67da8073b2a9728af19 -1 60 -4 120 create
    ```
 
-1. **[A]** Adapt the SBD configuration.
+6. **[A]** Adapt the SBD configuration.
 
    a. Open the SBD config file.
 
@@ -529,13 +546,16 @@ If you want to deploy resources by using the Azure CLI or the Azure portal, you
    [...]
    ```
 
-1. Create the `softdog` configuration file.
+    > [!NOTE]
+    > If the SBD_DELAY_START property value is set to "no", change the value to "yes". You must also check the SBD service file to ensure that the value of TimeoutStartSec is greater than the value of SBD_DELAY_START. For more information, see [SBD file configuraton](https://documentation.suse.com/sle-ha/15-SP5/html/SLE-HA-all/cha-ha-storage-protect.html#pro-ha-storage-protect-sbd-config)
+
+7. Create the `softdog` configuration file.
 
    ```bash
    echo softdog | sudo tee /etc/modules-load.d/softdog.conf
    ```
 
-1. Load the module.
+8. Load the module.
 
    ```bash
    sudo modprobe -v softdog
@@ -561,7 +581,7 @@ To create a service principal, do the following:
 2. Select **App registrations**.
 3. Select **New registration**.
 4. Enter a name for the registration, and then select **Accounts in this organization directory only**.
-5. For **Application type**, select **Web**, enter a sign-on URL (for example, *http://localhost*), and then select **Add**.  
+5. For **Application type**, select **Web**, enter a sign-on URL (for example, *<http://localhost>*), and then select **Add**.  
    The sign-on URL isn't used and can be any valid URL.
 6. Select **Certificates and secrets**, and then select **New client secret**.
 7. Enter a description for a new key, select **Two years**, and then select **Add**.
@@ -934,7 +954,7 @@ Make sure to assign the custom role to the service principal at all VM (cluster
    > The 'pcmk_host_map' option is required in the command only if the hostnames and the Azure VM names are *not* identical. Specify the mapping in the format *hostname:vm-name*.
    > Refer to the bold section in the following command.
 
-    #### [Managed identity](#tab/msi)
+#### [Managed identity](#tab/msi)
 
     ```bash
     # replace the bold strings with your subscription ID and resource group of the VM
@@ -947,7 +967,7 @@ Make sure to assign the custom role to the service principal at all VM (cluster
     sudo crm configure property stonith-timeout=900
     ```
 
-    #### [Service principal](#tab/spn)
+#### [Service principal](#tab/spn)
 
     ```bash
     # replace the bold strings with your subscription ID, resource group of the VM, tenant ID, service principal application ID and password
@@ -959,7 +979,7 @@ Make sure to assign the custom role to the service principal at all VM (cluster
    
     sudo crm configure property stonith-timeout=900
     ```
-   
+
     ---
 
     If you're using fencing device, based on service principal configuration, read [Change from SPN to MSI for Pacemaker clusters using Azure fencing](https://techcommunity.microsoft.com/t5/running-sap-applications-on-the/sap-on-azure-high-availability-change-from-spn-to-msi-for/ba-p/3609278) and learn how to convert to managed identity configuration.
@@ -985,20 +1005,21 @@ Azure offers [scheduled events](../../virtual-machines/linux/scheduled-events.md
    ```
 
    Minimum version requirements:
+
    - SLES 12 SP5: `resource-agents-4.3.018.a7fb5035-3.98.1`
    - SLES 15 SP1: `resource-agents-4.3.0184.6ee15eb2-150100.4.72.1`
    - SLES 15 SP2: `resource-agents-4.4.0+git57.70549516-150200.3.56.1`
    - SLES 15 SP3: `resource-agents-4.8.0+git30.d0077df0-150300.8.31.1`
    - SLES 15 SP4 and newer: `resource-agents-4.10.0+git40.0f4de473-150400.3.19.1`
 
-2. **[1]** Configure the resources in Pacemaker.
+1. **[1]** Configure the resources in Pacemaker.
 
    ```bash
    #Place the cluster in maintenance mode
    sudo crm configure property maintenance-mode=true
    ```
 
-3. **[1]** Set the pacemaker cluster health node strategy and constraint
+1. **[1]** Set the pacemaker cluster health node strategy and constraint
 
    ```bash
    sudo crm configure property node-health-strategy=custom
@@ -1010,15 +1031,15 @@ Azure offers [scheduled events](../../virtual-machines/linux/scheduled-events.md
    >
    > Don't define any other resources in the cluster starting with "health-", besides the resources described in the next steps of the documentation.
 
-4. **[1]** Set initial value of the cluster attributes.
+1. **[1]** Set initial value of the cluster attributes.
    Run for each cluster node. For scale-out environments including majority maker VM.
 
    ```bash
    sudo crm_attribute --node prod-cl1-0 --name '#health-azure' --update 0
    sudo crm_attribute --node prod-cl1-1 --name '#health-azure' --update 0
    ```
 
-5. **[1]** Configure the resources in Pacemaker.
+1. **[1]** Configure the resources in Pacemaker.
    Important: The resources must start with 'health-azure'.
 
    ```bash
@@ -1034,13 +1055,13 @@ Azure offers [scheduled events](../../virtual-machines/linux/scheduled-events.md
    >
    > WARNING: health-azure-events: unknown attribute 'allow-unhealthy-nodes'.
 
-6. Take the Pacemaker cluster out of maintenance mode
+1. Take the Pacemaker cluster out of maintenance mode
 
    ```bash
    sudo crm configure property maintenance-mode=false
    ```
 
-7. Clear any errors during enablement and verify that the health-azure-events resources have started successfully on all cluster nodes.
+1. Clear any errors during enablement and verify that the health-azure-events resources have started successfully on all cluster nodes.
 
    ```bash
    sudo crm resource cleanup
@@ -1051,7 +1072,7 @@ Azure offers [scheduled events](../../virtual-machines/linux/scheduled-events.md
    > [!NOTE]
    > After you've configured the Pacemaker resources for the azure-events agent, if you place the cluster in or out of maintenance mode, you might get warning messages such as:
    >
-   > WARNING: cib-bootstrap-options: unknown attribute 'hostName_ **hostname**'  
+   > WARNING: cib-bootstrap-options: unknown attribute 'hostName_**hostname**'  
    > WARNING: cib-bootstrap-options: unknown attribute 'azure-events_globalPullState'  
    > WARNING: cib-bootstrap-options: unknown attribute 'hostName_ **hostname**'  
    > These warning messages can be ignored.