Skip to content

Commit a728ce4

Browse files
committed
feat: add custom build of infiniband with ubuntu 24.04
Because Microsoft is only providing an image from 2022... :/ Signed-off-by: vsoch <vsoch@users.noreply.github.com>
1 parent 5d02845 commit a728ce4

File tree

5 files changed

+473
-0
lines changed

5 files changed

+473
-0
lines changed

azure/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -443,6 +443,10 @@ Now let's run lammps!
443443
# This should work (one node with ib and shared memory)
444444
flux run -o cpu-affinity=per-task -N1 -n 96 --env UCX_TLS=ib,sm --env UCX_NET_DEVICES=mlx5_ib0:1 lmp -v x 1 -v y 1 -v z 1 -in in.reaxff.hns -nocite
445445

446+
/opt/hpcx-v2.19-gcc-mlnx_ofed-ubuntu22.04-cuda12-x86_64/hpcx-rebuild/lib:/opt/hpcx-v2.19-gcc-mlnx_ofed-ubuntu22.04-cuda12-x86_64/hcoll/lib
447+
flux run -o cpu-affinity=per-task -N2 -n 192 --env OMPI_MPI_mca_coll_hcoll_enable=0 --env OMPI_MPI_mca_coll_ucc_enable=0 --env UCX_TLS=ib --env UCX_NET_DEVICES=mlx5_ib0:1 lmp -v x 1 -v y 1 -v z 1 -in in.reaxff.hns -nocite
448+
449+
446450
# -x UCC_LOG_LEVEL=debug -x UCC_TLS=ucp
447451
flux run -o cpu-affinity=per-task -N2 -n 192 --env UCC_LOG_LEVEL=info --env UCC_TLS=ucp --env UCC_CONFIG_FILE= -OMPI_MPI_mca_coll_ucc_enable=0 --env UCX_TLS=dc_x --env UCX_NET_DEVICES=mlx5_ib0:1 lmp -v x 1 -v y 1 -v z 1 -in in.reaxff.hns -nocite
448452
```

azure/build-ubuntu-24.04/Makefile

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
.PHONY: all
2+
all: init fmt validate build
3+
4+
.PHONY: init
5+
init:
6+
packer init .
7+
8+
.PHONY: fmt
9+
fmt:
10+
packer fmt .
11+
12+
.PHONY: validate
13+
validate:
14+
packer validate .
15+
16+
.PHONY: build
17+
build:
18+
packer build flux-usernetes.pkr.hcl

azure/build-ubuntu-24.04/README.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Build Packer Images
2+
3+
Note that I needed to do this build from a cloud shell, so clone and then:
4+
5+
```bash
6+
git clone https://github.com/converged-computing/flux-usernetes
7+
cd flux-usernetes/azure/build-ubuntu-24.04
8+
```
9+
10+
And install packer
11+
12+
```bash
13+
wget https://releases.hashicorp.com/packer/1.11.2/packer_1.11.2_linux_amd64.zip
14+
unzip packer_1.11.2_linux_amd64.zip
15+
mkdir -p ./bin
16+
mv ./packer ./bin/
17+
export PATH=$(pwd)/bin:$PATH
18+
```
19+
20+
Get your account information for azure as follows:
21+
22+
```bash
23+
az account show
24+
```
25+
26+
And export variables in the following format. Note that the resource group needs to actually exist - I created mine in the console UI.
27+
28+
```bash
29+
export AZURE_SUBSCRIPTION_ID=xxxxxxxxx
30+
export AZURE_TENANT_ID=xxxxxxxxxxx
31+
export AZURE_RESOURCE_GROUP_NAME=packer-testing
32+
```
33+
34+
Then build!
35+
36+
```bash
37+
make
38+
```

0 commit comments

Comments
 (0)