Merge pull request #313427 from dennispadia/dp-suse-scaleout

Changes in HANA Scale-out document

Author: denrea

articles/sap/workloads/high-availability-guide-suse-pacemaker.md

@@ -8,7 +8,7 @@ ms.service: sap-on-azure
 ms.subservice: sap-vm-workloads
 ms.topic: article
 ms.custom: devx-track-azurepowershell, linux-related-content
-ms.date: 03/12/2026
+ms.date: 03/19/2026
 ms.author: radeltch
 # Customer intent: "As a system administrator, I want to set up Pacemaker with fencing on SUSE Linux Enterprise Server in Azure, so that I can ensure high availability and reliability for my applications running in the cloud."
 ---
@@ -871,37 +871,52 @@ Make sure to assign the custom role to the service principal at all VM (cluster
 
       ```bash
       sudo crm cluster init
-      # ! NTP is not configured to start at system boot.
-      # Do you want to continue anyway (y/n)? y
-      # /root/.ssh/id_rsa already exists - overwrite (y/n)? n
-      # Address for ring0 [10.0.0.6] Select Enter
-      # Port for ring0 [5405] Select Enter
-      # SBD is already configured to use /dev/disk/by-id/scsi-36001405639245768818458b930abdf69;/dev/disk/by-id/scsi-36001405afb0ba8d3a3c413b8cc2cca03;/dev/disk/by-id/scsi-36001405f88f30e7c9684678bc87fe7bf - overwrite (y/n)? n
+      
+      # INFO: Detected "microsoft-azure" platform
+      # INFO: Loading "default" profile from /etc/crm/profiles.yml
+      # INFO: Loading "microsoft-azure" profile from /etc/crm/profiles.yml
+      # INFO: The user 'hacluster' will have the login shell configuration changed to /bin/bash 
+      # Continue (y/n)? y
+      # INFO: Address for ring0 [10.0.0.6] Select Enter
+      # INFO: Port for ring0 [5405] Select Enter
+      # INFO: Do you wish to use SBD (y/n)? y
+      # INFO: SBD is already configured to use /dev/disk/by-id/scsi-36001405639245768818458b930abdf69;/dev/disk/by-id/scsi-36001405afb0ba8d3a3c413b8cc2cca03;/dev/disk/by-id/scsi-36001405f88f30e7c9684678bc87fe7bf - overwrite (y/n)? n
       # Do you wish to configure an administration IP (y/n)? n
+      # INFO: Do you wish to configure a virtual IP address (y/n)? n
       ```
 
     - If you're *not* using SBD devices for fencing:
 
       ```bash
       sudo crm cluster init
-      # ! NTP is not configured to start at system boot.
-      # Do you want to continue anyway (y/n)? y
-      # /root/.ssh/id_rsa already exists - overwrite (y/n)? n
-      # Address for ring0 [10.0.0.6] Select Enter
-      # Port for ring0 [5405] Select Enter
-      # Do you wish to use SBD (y/n)? n
+
+      # INFO: Detected "microsoft-azure" platform
+      # INFO: Loading "default" profile from /etc/crm/profiles.yml
+      # INFO: Loading "microsoft-azure" profile from /etc/crm/profiles.yml
+      # INFO: The user 'hacluster' will have the login shell configuration changed to /bin/bash 
+      # Continue (y/n)? y
+      # INFO: Address for ring0 [10.0.0.6] Select Enter
+      # INFO: Port for ring0 [5405] Select Enter
+      # INFO: Do you wish to use SBD (y/n)? n
       # WARNING: Not configuring SBD - STONITH will be disabled.
       # Do you wish to configure an administration IP (y/n)? n
+      # INFO: Do you wish to configure a virtual IP address (y/n)? n
       ```
 
 14. **[2]** Add the node to the cluster.
 
     ```bash
     sudo crm cluster join
-    # ! NTP is not configured to start at system boot.
-    # Do you want to continue anyway (y/n)? y
-    # IP address or hostname of existing node (for example, 192.168.1.1) []10.0.0.6
-    # /root/.ssh/id_rsa already exists - overwrite (y/n)? n
+    # INFO: IP address or hostname of existing node (e.g.: 192.168.1.1) []10.0.0.6
+    # INFO: The user 'hacluster' will have the login shell configuration changed to /bin/bash
+    # INFO: Continue (y/n)? y
+    # INFO: Generating SSH key for hacluster
+    # INFO: Configuring SSH passwordless with [email protected]
+    # INFO: Configuring csync2...done
+    # INFO: Merging known_hosts
+    # INFO: Probing for new partitions...done
+    # INFO: Address for ring0 [10.0.0.7] Select Enter
+    # INFO: Done (log saved to /var/log/crmsh/crmsh.log)
     ```
 
 15. **[A]** Change the hacluster password to the same password.
@@ -919,6 +934,7 @@ Make sure to assign the custom role to the service principal at all VM (cluster
     a. Check the following section in the file and adjust, if the values aren't there or are different. Be sure to change the token to 30000 to allow memory-preserving maintenance. For more information, see the "Maintenance for virtual machines in Azure" article for [Linux][virtual-machines-linux-maintenance] or [Windows][virtual-machines-windows-maintenance].
 
     ```text
+    {
     [...]
       token:          30000
       token_retransmits_before_loss_const: 10
@@ -968,73 +984,88 @@ Make sure to assign the custom role to the service principal at all VM (cluster
 1. **[1]** If you're using an SBD device (iSCSI target server or Azure shared disk) as a fencing device, run the following commands. Enable the use of a fencing device, and set the fence delay.
 
    ```bash
-   sudo crm configure property stonith-timeout=210
-   sudo crm configure property stonith-enabled=true
-   
    # List the resources to find the name of the SBD device
    sudo crm resource list
    sudo crm resource stop stonith-sbd
    sudo crm configure delete stonith-sbd
+
    sudo crm configure primitive stonith-sbd stonith:external/sbd \
       params pcmk_delay_max="15" \
       op monitor interval="600" timeout="15"
-   ```
 
-1. **[1]** If you're using an Azure fence agent for fencing, run the following commands. After assigning roles to both cluster nodes, you can configure the fencing devices in the cluster.
+   # For SAP HANA scale-out only, configure stonith-sbd using following command
+   sudo crm configure primitive stonith-sbd stonith:external/sbd \
+      params pcmk_action_limit=-1 \
+      op monitor interval="600" timeout="15"
 
-   ```bash
+   sudo crm configure property stonith-timeout=210
    sudo crm configure property stonith-enabled=true
-   sudo crm configure property concurrent-fencing=true
    ```
 
+1. **[1]** If you're using an Azure fence agent for fencing, run the following commands. After assigning roles to both cluster nodes, you can configure the fencing devices in the cluster.
+
    > [!NOTE]
    > The 'pcmk_host_map' option is required in the command only if the hostnames and the Azure VM names are *not* identical. Specify the mapping in the format *hostname:vm-name*.
 
-#### [Managed identity](#tab/msi)
+   #### [Managed identity](#tab/msi)
 
    ```bash
    # Adjust the command with your subscription ID and resource group of the VM
-    
    sudo crm configure primitive rsc_st_azure stonith:fence_azure_arm \
    params msi=true subscriptionId="subscription ID" resourceGroup="resource group" \
    pcmk_monitor_retries=4 pcmk_action_limit=3 power_timeout=240 pcmk_reboot_timeout=900 pcmk_delay_max=15 pcmk_host_map="prod-cl1-0:prod-cl1-0-vm-name;prod-cl1-1:prod-cl1-1-vm-name" \
    meta failure-timeout=120s \
    op monitor interval=3600 timeout=120
-       
-   sudo crm configure property stonith-timeout=900
+
+   # For SAP HANA scale-out only, configure fence_azure_arm using following command
+   sudo crm configure primitive rsc_st_azure stonith:fence_azure_arm \
+   params msi=true subscriptionId="subscription ID" resourceGroup="resource group" \
+   pcmk_monitor_retries=4 pcmk_action_limit=-1 power_timeout=240 pcmk_reboot_timeout=900 pcmk_host_map="prod-cl1-0:prod-cl1-0-vm-name;prod-cl1-1:prod-cl1-1-vm-name" \
+   meta failure-timeout=120s \
+   op monitor interval=3600 timeout=120
    ```
 
-#### [Service principal](#tab/spn)
+   #### [Service principal](#tab/spn)
 
    ```bash
    # Adjust the command with your subscription ID, resource group of the VM, tenant ID, service principal application ID and password
-   
    sudo crm configure primitive rsc_st_azure stonith:fence_azure_arm \
    params subscriptionId="subscription ID" resourceGroup="resource group" tenantId="tenant ID" login="application ID" passwd="password" \
    pcmk_monitor_retries=4 pcmk_action_limit=3 power_timeout=240 pcmk_reboot_timeout=900 pcmk_delay_max=15 pcmk_host_map="prod-cl1-0:prod-cl1-0-vm-name;prod-cl1-1:prod-cl1-1-vm-name" \
    meta failure-timeout=120s \
    op monitor interval=3600 timeout=120
-   
-   sudo crm configure property stonith-timeout=900
+
+   # For SAP HANA scale-out only, configure fence_azure_arm using following command
+   sudo crm configure primitive rsc_st_azure stonith:fence_azure_arm \
+   params subscriptionId="subscription ID" resourceGroup="resource group" tenantId="tenant ID" login="application ID" passwd="password" \
+   pcmk_monitor_retries=4 pcmk_action_limit=-1 power_timeout=240 pcmk_reboot_timeout=900 pcmk_host_map="prod-cl1-0:prod-cl1-0-vm-name;prod-cl1-1:prod-cl1-1-vm-name" \
+   meta failure-timeout=120s \
+   op monitor interval=3600 timeout=120
    ```
 
----
+    ---
+
+   ```bash
+   sudo crm configure property stonith-enabled=true
+   sudo crm configure property concurrent-fencing=true
+   sudo crm configure property stonith-timeout=900
+   ```
 
 If you're using fencing device, based on service principal configuration, read [Change from SPN to MSI for Pacemaker clusters using Azure fencing](https://techcommunity.microsoft.com/t5/running-sap-applications-on-the/sap-on-azure-high-availability-change-from-spn-to-msi-for/ba-p/3609278) and learn how to convert to managed identity configuration.
 
-   > [!IMPORTANT]
-   > The monitoring and fencing operations are deserialized. As a result, if there's a longer-running monitoring operation and simultaneous fencing event, there's no delay to the cluster failover because the monitoring operation is already running.
+> [!IMPORTANT]
+> The monitoring and fencing operations are deserialized. As a result, if there's a longer-running monitoring operation and simultaneous fencing event, there's no delay to the cluster failover because the monitoring operation is already running.
 
-   > [!TIP]
-   >The Azure fence agent requires outbound connectivity to the public endpoints, as documented, along with possible solutions, in [Public endpoint connectivity for VMs using standard ILB](./high-availability-guide-standard-load-balancer-outbound-connections.md).  
+> [!TIP]
+> The Azure fence agent requires outbound connectivity to the public endpoints, as documented, along with possible solutions, in [Public endpoint connectivity for VMs using standard ILB](./high-availability-guide-standard-load-balancer-outbound-connections.md).  
 
 ## Configure Pacemaker for Azure scheduled events
 
-Azure offers [scheduled events](/azure/virtual-machines/linux/scheduled-events). Scheduled events are provided via the metadata service and allow time for the application to prepare for such events. 
+Azure offers [scheduled events](/azure/virtual-machines/linux/scheduled-events). Scheduled events are provided via the metadata service and allow time for the application to prepare for such events.
 
 Resource agent [azure-events-az](https://github.com/ClusterLabs/resource-agents/pull/1161) monitors for scheduled Azure events. If events are detected and the resource agent determines that another cluster node is available, it sets a node-level health attribute `#health-azure` to `-1000000`. 
 
-When this special cluster health attribute is set for a node, the node is considered unhealthy by the cluster and all resources are migrated away from the affected node. The location constraint ensures resources with name starting with ‘health-‘ are excluded, as the agent needs to run in this unhealthy state. Once the affected cluster node is free of running cluster resources, scheduled event can execute its action, such as restart, without risk to running resources. 
+When this special cluster health attribute is set for a node, the node is considered unhealthy by the cluster and all resources are migrated away from the affected node. The location constraint ensures resources with name starting with ‘health-‘ are excluded, as the agent needs to run in this unhealthy state. Once the affected cluster node is free of running cluster resources, scheduled event can execute its action, such as restart, without risk to running resources.
 
 The `#heath-azure` attribute is set back to `0` on pacemaker startup once all events have been processed, marking the node as healthy again.