|
1 | 1 | # Cray Programming Environment (CPE) |
2 | 2 |
|
3 | | -!!! warning |
4 | | - you don't want to use this |
| 3 | +The Cray Programming Environment (CPE) is a suite of software: programming environments, compilers, libraries and tools. |
5 | 4 |
|
6 | | -!!! mlp |
7 | | - The CPE is not provided on the machine learning platform. |
| 5 | +The CPE is provided to users on Alps as containers. |
8 | 6 |
|
9 | | -!!! cwp |
10 | | - The CPE is not provided on the climate and weather platform |
| 7 | +!!! info "CPE is the cray modules" |
| 8 | + The familiar modules that provide the `Prgenv-gnu` and `Prgenv-cray` programming environments, and packages like `cray-python` and `cray-fftw`, that will be familiar from the old Piz Daint system. |
| 9 | + |
| 10 | +!!! warning "CPE is not supported on Alps" |
| 11 | + The Cray modules were provided on the old Daint system, and CSCS supported their use and provided software built on top of them. |
| 12 | + |
| 13 | + **Alps is a big change - the CPE modules are not provide as officially supported software.** |
| 14 | + They are provided _as is_ to users who still need to use them, however CSCS will not be able to provide detailed support for issues that arise when using them. |
| 15 | + |
| 16 | + The recommended method for building and running software is to use [uenv][ref-uenv] or [containers][ref-container-engine]. |
| 17 | + |
| 18 | +## CPE in a container |
| 19 | + |
| 20 | +[](){#ref-cpe-versions} |
| 21 | +### Available versions |
| 22 | + |
| 23 | +The `PrgEnv-gnu` and `PrgEnv-cray` programming environments are provided as separate containers, instead of having both in one container, named `gnu-$version` and `cray-$version` respectively. |
| 24 | +The `version` is the CPE version in the container. |
| 25 | + |
| 26 | +| | `zen2` | `gh200` | |
| 27 | +|-----------------|----------|---------| |
| 28 | +| `cpe-gnu-24.7` | ❌ | ✅ | |
| 29 | +| `cpe-cray-24.7` | ❌ | ✅ | |
| 30 | + |
| 31 | +!!! warning "only available on gh200" |
| 32 | + The CPE container is only provided on systems with the "[container engine][ref-container-engine]" container runtime, which is currently the Grace-Hopper systems [Daint][ref-cluster-daint], [Clariden][ref-cluster-clariden] and [Santis][ref-cluster-santis]. |
| 33 | + |
| 34 | +### How to use |
| 35 | + |
| 36 | +!!! info "Before you start" |
| 37 | + To use the CPE containers as documented below, you need to set the `EDF_PATH` environment variable. |
| 38 | + ```console |
| 39 | + export EDF_PATH=/capstor/scratch/cscs/anfink/shared/cpe/edf:$EDF_PATH |
| 40 | + ``` |
| 41 | + Note that the current location of the EDF files is temporary, to work around a file system bug. |
| 42 | + The environment variable __will not be required__ once this issue is fixed. |
| 43 | + |
| 44 | +To start a session with the CPE (see [available-versions][ref-cpe-versions] above): |
| 45 | +```console |
| 46 | +$ srun --environment=cpe-cray-24.07 --pty bash |
| 47 | +``` |
| 48 | +Once the container starts up you can directly use the programming environment, because there will be modules loaded by default at startup. |
| 49 | + |
| 50 | +```console |
| 51 | +$ srun -p debug --environment=cpe-cray-24.07 --pty bash |
| 52 | +$ module list |
| 53 | + |
| 54 | +Currently Loaded Modules: |
| 55 | + 1) craype-arm-grace 4) cce/18.0.0 7) cray-mpich/8.1.30 |
| 56 | + 2) craype-network-ofi 5) craype/2.7.32 8) cuda/12.6 |
| 57 | + 3) xpmem/2.9.6 6) PrgEnv-cray/8.5.0 9) craype-accel-nvidia90 |
| 58 | + |
| 59 | + |
| 60 | +$ module avail |
| 61 | + |
| 62 | +---- /opt/cray/pe/lmod/modulefiles/mpi/crayclang/17.0/ofi/1.0/cray-mpich/8.0 ---- |
| 63 | + cray-hdf5-parallel/1.14.3.1 cray-parallel-netcdf/1.12.3.13 |
| 64 | + |
| 65 | +---------- /opt/cray/pe/lmod/modulefiles/comnet/crayclang/17.0/ofi/1.0 ---------- |
| 66 | + cray-mpich-abi/8.1.30 cray-mpich/8.1.30 (L) |
| 67 | + |
| 68 | +------------- /opt/cray/pe/lmod/modulefiles/compiler/crayclang/17.0 ------------- |
| 69 | + cray-hdf5/1.14.3.1 cray-libsci/24.07.0 |
| 70 | + |
| 71 | +------------------ /opt/cray/pe/lmod/modulefiles/mix_compilers ------------------ |
| 72 | + cce-mixed/18.0.0 |
| 73 | + |
| 74 | +---------------- /opt/cray/pe/lmod/modulefiles/cpu/arm-grace/1.0 ---------------- |
| 75 | + cray-fftw/3.3.10.8 |
| 76 | + |
| 77 | +------------- /opt/cray/pe/lmod/modulefiles/craype-targets/default -------------- |
| 78 | + craype-accel-amd-gfx908 craype-hugepages16M craype-network-none |
| 79 | + craype-accel-amd-gfx90a craype-hugepages1G craype-network-ofi (L) |
| 80 | + craype-accel-amd-gfx940 craype-hugepages256M craype-network-ucx |
| 81 | + craype-accel-amd-gfx942 craype-hugepages2G craype-x86-genoa |
| 82 | + craype-accel-host craype-hugepages2M craype-x86-milan-x |
| 83 | + craype-accel-nvidia70 craype-hugepages32M craype-x86-milan |
| 84 | + craype-accel-nvidia80 craype-hugepages4M craype-x86-rome |
| 85 | + craype-accel-nvidia90 (L) craype-hugepages512M craype-x86-spr-hbm |
| 86 | + craype-arm-grace (L) craype-hugepages64M craype-x86-spr |
| 87 | + craype-hugepages128M craype-hugepages8M craype-x86-trento |
| 88 | + |
| 89 | +---------------------- /opt/cray/pe/lmod/modulefiles/core ----------------------- |
| 90 | + PrgEnv-cray/8.5.0 (L) cray-libsci_acc/24.07.0 cray-python/3.11.7 |
| 91 | + cce/18.0.0 (L) cray-pmi/6.1.15.19 craype/2.7.32 (L) |
| 92 | + |
| 93 | +----------------------------- /opt/cray/modulefiles ----------------------------- |
| 94 | + xpmem/2.9.6 (L) |
| 95 | + |
| 96 | +----------------------------- /opt/cscs/modulefiles ----------------------------- |
| 97 | + cuda/12.6 (L) |
| 98 | + |
| 99 | +$ CC --version |
| 100 | +Cray clang version 18.0.0 (0e4696aa65fa9549bd5e19c216678cc98185b0f7) |
| 101 | +Target: aarch64-unknown-linux-gnu |
| 102 | +Thread model: posix |
| 103 | +InstalledDir: /opt/cray/pe/cce/18.0.0/cce-clang/aarch64/share/../bin |
| 104 | +``` |
| 105 | + |
| 106 | +The recommended way of using CPE in a container is to start the container, and use `$SCRATCH` and `$STORE` to interact with persistent data. Please remember that any data that is written to a directory that is not mounted from the host system will be lost, after the container stops. |
| 107 | + |
| 108 | +!!! note |
| 109 | + By default, the paths `/capstor`, `/iopsstor` are mounted to the same paths inside the container. |
| 110 | + |
| 111 | +Additionally `/users` will be mounted at `/users.host`, so you can access data in your home folder, but with a slightly different path. This is on purpose, and you can override this behaviour by writing your own [EDF file][ref-ce-edf-reference], especially using the key `base_environment`, referencing the predefined CPE environment files and override what you would like to change. |
0 commit comments