Skip to content

document CPE in a container #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 23, 2025
Merged

document CPE in a container #99

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1776503
document CPE in a container
finkandreas Apr 23, 2025
ce3dbe6
typo
finkandreas Apr 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions docs/build-install/cpe.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,90 @@

!!! cwp
The CPE is not provided on the climage and weather platform

## CPE in a container
!!! info
Currently it is mandatory to export the EDF path manually
```
export EDF_PATH=/capstor/scratch/cscs/anfink/shared/cpe/edf:$EDF_PATH
```

To start a container with the cray programming environment, there are pre-defined EDF toml files. Currently they are stored in `/capstor/scratch/cscs/anfink/shared/cpe/edf`, but this will change in the future to a different location.
Every toml file in this edf path is a programming environment that can be started
```
[daint][anfink@daint-ln001 ~]$ ls -lh /capstor/scratch/cscs/anfink/shared/cpe/edf
total 8.0K
-rw-r--r--+ 1 anfink csstaff 175 Apr 23 11:48 cpe-cray-24.07.toml
-rw-r--r--+ 1 anfink csstaff 174 Apr 23 11:32 cpe-gnu-24.07.toml
```
Following the naming scheme from the output, you can spawn a container with
```
srun --environment=cpe-cray-24.07 --pty bash
```
Once the container starts up you can directly use the programming environment, because there will be modules loaded by default at startup.
```
[daint][anfink@daint-ln001 ~]$ srun -p debug --environment=cpe-gnu-24.07 --pty bash
[daint][anfink@nid005417 /]$ module list

Currently Loaded Modules:
1) craype-arm-grace 2) craype-network-ofi 3) xpmem/2.9.6 4) gcc-native/13.2 5) craype/2.7.32 6) PrgEnv-gnu/8.5.0 7) cray-mpich/8.1.30 8) cuda/12.6 9) craype-accel-nvidia90



[daint][anfink@nid005417 /]$ module avail

----------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/mpi/gnu/12.0/ofi/1.0/cray-mpich/8.0 -----------------------------------------------------------------------------------------------------------------------------------------------------
cray-hdf5-parallel/1.14.3.1 cray-parallel-netcdf/1.12.3.13

----------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/comnet/gnu/12.0/ofi/1.0 -----------------------------------------------------------------------------------------------------------------------------------------------------------
cray-mpich-abi/8.1.30 cray-mpich/8.1.30 (L)

---------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/mix_compilers ----------------------------------------------------------------------------------------------------------------------------------------------------------------
gcc-native-mixed/13.2

-------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/compiler/gnu/12.0 --------------------------------------------------------------------------------------------------------------------------------------------------------------
cray-hdf5/1.14.3.1 cray-libsci/24.07.0

-------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/cpu/arm-grace/1.0 --------------------------------------------------------------------------------------------------------------------------------------------------------------
cray-fftw/3.3.10.8

----------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/craype-targets/default ------------------------------------------------------------------------------------------------------------------------------------------------------------
craype-accel-amd-gfx908 craype-accel-amd-gfx942 craype-accel-nvidia80 craype-hugepages128M craype-hugepages256M craype-hugepages32M craype-hugepages64M craype-network-ofi (L) craype-x86-milan-x craype-x86-spr-hbm
craype-accel-amd-gfx90a craype-accel-host craype-accel-nvidia90 (L) craype-hugepages16M craype-hugepages2G craype-hugepages4M craype-hugepages8M craype-network-ucx craype-x86-milan craype-x86-spr
craype-accel-amd-gfx940 craype-accel-nvidia70 craype-arm-grace (L) craype-hugepages1G craype-hugepages2M craype-hugepages512M craype-network-none craype-x86-genoa craype-x86-rome craype-x86-trento

-------------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/core ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
PrgEnv-gnu/8.5.0 (L) cray-libsci_acc/24.07.0 cray-pmi/6.1.15.19 cray-python/3.11.7 craype/2.7.32 (L) gcc-native/13.2 (L)

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/modulefiles ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
xpmem/2.9.6 (L)

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cscs/modulefiles ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cuda/12.6 (L)

Where:
L: Module is loaded

If the avail list is too long consider trying:

"module --default avail" or "ml -d av" to just list the default modules.
"module overview" or "ml ov" to display the number of modules for each name.

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".


[daint][anfink@nid005417 /]$ CC --version
g++-13 (SUSE Linux) 13.3.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[daint][anfink@nid005417 /]$
```

The recommended way of using CPE in a container is to start the container, and use `$SCRATCH` and `$STORE` to interact with persistent data. Please remember that any data that is written to a directory that is not mounted from the host system will be lost, after the container stops.

By default, the paths `/capstor`, `/iopsstor` are mounted to the same paths inside the container.

Additionally `/users` will be mounted at `/users.host`, so you can access data in your home folder, but with a slightly different path. This is on purpose, and you can override this behaviour by writing your own [EDF file][ref-ce-edf-reference], especially using the key `base_environment`, referencing the predefined CPE environment files and override what you would like to change.