Skip to content

Commit 2dba083

Browse files
committed
add LUMI configuration
1 parent e6c4127 commit 2dba083

File tree

2 files changed

+80
-0
lines changed

2 files changed

+80
-0
lines changed
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
#!/bin/bash
2+
3+
# Author: Samuel Omlin, CSCS (omlins)
4+
#
5+
# Description: Definition of site specific variables and call of JUHPC.
6+
# Site: LUMI, EuroHPC JU
7+
# Base: craype
8+
9+
10+
# module load LUMI/22.08
11+
# module load partition/G
12+
# module load rocm/5.3.3
13+
# module load cray-hdf5-parallel
14+
15+
# module load CrayEnv
16+
# module switch PrgEnv-cray PrgEnv-gnu #PrgEnv-amd
17+
# module load rocm craype-accel-nvidia90
18+
# module load cray-hdf5-parallel
19+
# module list
20+
21+
# Load required modules (including correct CPU and GPU target modules)
22+
module load LUMI
23+
module load partition/G # loads CPU and GPU target modules (craype-x86-trento, craype-accel-amd-gfx90a)
24+
module load cpeGNU # LUMI-wrapper for PrgEnv-gnu
25+
module load rocm
26+
module load cray-hdf5-parallel
27+
module list
28+
29+
# Environment variables for HPC key packages that require system libraries that require system libraries (MPI.jl, CUDA.jl, HDF5.jl and ADIOS2.jl)
30+
export JUHPC_ROCM_HOME=$ROCM_PATH
31+
export JUHPC_MPI_VENDOR="cray"
32+
export JUHPC_MPI_EXEC="srun"
33+
export JUHPC_HDF5_HOME=$HDF5_DIR
34+
35+
36+
# Call JUHPC
37+
JUHPC_SETUP_INSTALLDIR=$SCRATCH/${HOSTNAME%%[0-9]*}/juhpc_setup #SCRATCH is assumed to be defined in ~/.bashrc, e.g., SCRATCH=/scratch/project_465000105/$USER
38+
JULIAUP_INSTALLDIR="\$SCRATCH/\${HOSTNAME%%[0-9]*}/juliaup"
39+
VERSION="v0.1.0"
40+
#wget https://raw.githubusercontent.com/JuliaParallel/JUHPC/$VERSION/juhpc -O /tmp/juhpc
41+
wget https://raw.githubusercontent.com/JuliaParallel/JUHPC/main/juhpc -O /tmp/juhpc
42+
bash -l /tmp/juhpc $JUHPC_SETUP_INSTALLDIR $JULIAUP_INSTALLDIR
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#!/bin/bash
2+
3+
# Variable set in craype_config
4+
JUHPC_SETUP_INSTALLDIR=$SCRATCH/${HOSTNAME%%[0-9]*}/juhpc_setup
5+
6+
# Load required modules (including correct CPU and GPU target modules)
7+
module load LUMI
8+
module load partition/G # loads CPU and GPU target modules (craype-x86-trento, craype-accel-amd-gfx90a)
9+
module load cpeGNU # LUMI-wrapper for PrgEnv-gnu
10+
module load rocm
11+
module load cray-hdf5-parallel
12+
module list
13+
14+
# Activate the HPC setup environment variables
15+
. $JUHPC_SETUP_INSTALLDIR/activate
16+
17+
# Call juliaup to install juliaup and latest julia on scratch
18+
juliaup
19+
20+
# Call juliaup to see its options
21+
juliaup
22+
23+
# Call julia Pkg
24+
julia -e 'using Pkg; Pkg.status()'
25+
26+
# Add AMDGPU.jl
27+
julia -e 'using Pkg; Pkg.add("AMDGPU"); using AMDGPU; AMDGPU.versioninfo()'
28+
29+
# Add MPI.jl
30+
julia -e 'using Pkg; Pkg.add("MPI"); using MPI; MPI.versioninfo()'
31+
32+
# Add HDF5.jl
33+
julia -e 'using Pkg; Pkg.add("HDF5"); using HDF5; @show HDF5.has_parallel()'
34+
35+
# Test AMDGPU-aware MPI
36+
julia -e 'using Pkg; Pkg.add(["ImplicitGlobalGrid", "ParallelStencil"]);'
37+
cd ~/rocmaware
38+
MPICH_GPU_SUPPORT_ENABLED=1 IGG_ROCMAWARE_MPI=1 srun --time=00:09:00 -pdev-g -Aproject_465000557 -N2 -n2 --gpus-per-node=8 julia -O3 --check-bounds=no diffusion3D.jl

0 commit comments

Comments
 (0)