You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> Scale sets have limitations on size (currently 1000 VMs). To scale larger than that, you must create multiple scale sets and assign them to different nodearrays.
41
+
> Scale sets have limitations on size (currently 1,000 VMs). To scale larger than that size, you must create multiple scale sets and assign them to different node arrays.
Copy file name to clipboardExpand all lines: articles/cyclecloud/how-to/hb-hc-best-practices.md
+11-11Lines changed: 11 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,31 +2,31 @@
2
2
title: HB/HC Cluster Best Practices
3
3
description: Best practices for using Azure CycleCloud with HB and HC series Virtual Machines.
4
4
author: anhoward
5
-
ms.date: 06/11/2019
5
+
ms.date: 07/01/2025
6
6
ms.author: anhoward
7
7
---
8
8
9
-
# Best Practices for using HB and HC VMs
9
+
# Best practices for using HB and HC VMs
10
10
11
11
## Overview
12
12
13
-
The [H-series virtual machines](/azure/virtual-machines/windows/sizes-hpc) (VMs) are the latest HPC offerings on Azure. HB-series VMs offer 60-core AMD EPYC processors, optimized for running applications with high memory-bandwidth requirements, such as explicit finite element analysis, fluid dynamics, and weather modeling. The HC-series VMs have 44-core Intel Xeon Skylake processors and are optimized for applications requiring intensive CPU calculations, like molecular dynamics and implicit finite element analysis. HB and HC VMs feature 100Gb/s EDR InfiniBand and support the latest MPI types and versions. The [Scaling HPC Applications Guide](/azure/virtual-machines/workloads/hpc/compiling-scaling-applications) has more information on how to scale HPC applications on HB and HC VMs.
13
+
The [H-series virtual machines](/azure/virtual-machines/windows/sizes-hpc) (VMs) are the latest HPC offerings on Azure. HB-series VMs offer 60-core AMD EPYC processors and are optimized for running applications with high memory-bandwidth requirements, such as explicit finite element analysis, fluid dynamics, and weather modeling. The HC-series VMs have 44-core Intel Xeon Skylake processors and are optimized for applications requiring intensive CPU calculations, like molecular dynamics and implicit finite element analysis. HB and HC VMs feature 100-Gb/s EDR InfiniBand and support the latest MPI types and versions. For more information on how to scale HPC applications on HB and HC VMs, see the [Scaling HPC Applications Guide](/azure/virtual-machines/workloads/hpc/compiling-scaling-applications).
14
14
15
-
Azure CycleCloud supports the new H-series VMs out of the box, but for the best experience and performance, follow the guidelines and best practices on this page.
15
+
Azure CycleCloud supports the new H-series VMs, but for the best experience and performance, follow the guidelines and best practices in this article.
16
16
17
-
## CentOS 7.6 HPC Marketplace Image
17
+
## CentOS 7.6 HPC Marketplace image
18
18
19
-
The CentOS 7.6 HPC Marketplace image contains all of the drivers to enable the InfiniBand interface as well as pre-compiled versions of all of the common MPI variants installed in */opt*. For details on what exactly the image has to offer see [this blog post](https://techcommunity.microsoft.com/t5/Azure-Compute/CentOS-HPC-VM-Image-for-SR-IOV-enabled-Azure-HPC-VMs/ba-p/665557).
19
+
The CentOS 7.6 HPC Marketplace image contains all of the drivers to enable the InfiniBand interface as well as precompiled versions of all of the common MPI variants installed in */opt*. For details on what the image offers, see [this blog post](https://techcommunity.microsoft.com/t5/Azure-Compute/CentOS-HPC-VM-Image-for-SR-IOV-enabled-Azure-HPC-VMs/ba-p/665557).
20
20
21
21
To use the CentOS 7.6 HPC image when creating your cluster, check the **Custom Image** box on the **Advanced Settings** parameter and enter the value `OpenLogic:CentOS-HPC:7.6:latest`.
In order to support the older H16r VM series and keep cluster head nodes locked to the same version of CentOS, the default "Cycle CentOS 7" image in the Base OS dropdown deploys CentOS 7.4. While this is fine for most VM series, HB/HC VMs require CentOS 7.6 or newer and a different Mellanox driver.
25
+
To support the older H16r VM series and keep cluster head nodes locked to the same version of CentOS, the default "Cycle CentOS 7" image in the Base OS dropdown deploys CentOS 7.4. While this version works for most VM series, HB and HC VMs require CentOS 7.6 or newer and a different Mellanox driver.
26
26
27
-
## Disable SElinux in CycleCloud < 7.7.4
27
+
## Disable SElinux in CycleCloud versions earlier than 7.7.4
28
28
29
-
By default, SElinux only considers */root* and */home* to be valid paths for home directories. Any users with home directories outside of these paths cause SElinux to block SSH from using any SSH keypairs in the user's home directory. In CycleCloud clusters, user home directories are created in */shared/home*. While CycleCloud versions newer than 7.7.4 automatically set the */shared/home* path as a valid SElinux homedir context, older versions don't support this. In order to make sure SSH works properly for users on the cluster, you need to disable SElinux in the cluster template:
29
+
By default, SElinux only considers `/root` and `/home` to be valid paths for home directories. If users have home directories outside of these paths, SElinux blocks SSH from using any SSH keypairs in the user's home directory. In CycleCloud clusters, you create user home directories in `/shared/home`. While CycleCloud versions newer than 7.7.4 automatically set the `/shared/home` path as a valid SElinux homedir context, older versions don't support this feature. To make sure SSH works properly for users on the cluster, disable SElinux in the cluster template:
30
30
```ini
31
31
[[node defaults]]
32
32
[[[configuration]]]
@@ -35,7 +35,7 @@ By default, SElinux only considers */root* and */home* to be valid paths for hom
35
35
36
36
## Running MPI jobs with Slurm
37
37
38
-
MPI jobs running on HB/HC VMs need to run in the same VM Scaleset (VMSS). To ensure proper autoscale placement of VMs for MPI jobs running with Slurm, make sure to set the following attribute in your cluster template:
38
+
MPI jobs running on HB or HC VMs need to run in the same virtual machine scale set. To ensure proper autoscale placement of VMs for MPI jobs running with Slurm, set the following attribute in your cluster template:
39
39
40
40
```ini
41
41
[[nodearray execute]]
@@ -46,7 +46,7 @@ Azure.Overprovision = true
46
46
47
47
## Getting pkeys for use with OpenMPI and MPICH
48
48
49
-
Some MPI variants require you to specify the InfiniBand PKEY when running the job. The following Bash function can be used to determine the PKEY:
49
+
Some MPI variants require you to specify the InfiniBand PKEY when running the job. Use the following Bash function to determine the PKEY:
0 commit comments