Skip to content

Commit 6eed060

Browse files
bcummingbcumming
authored
refactor the cpe docs (#101)
refactor the cpe docs
1 parent 9f008be commit 6eed060

File tree

3 files changed

+78
-69
lines changed

3 files changed

+78
-69
lines changed

‎docs/build-install/cpe.md‎

Lines changed: 77 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,97 +1,111 @@
11
# Cray Programming Environment (CPE)
22

3-
!!! warning
4-
you don't want to use this
3+
The Cray Programming Environment (CPE) is a suite of software: programming environments, compilers, libraries and tools.
54

6-
!!! mlp
7-
The CPE is not provided on the machine learning platform.
5+
The CPE is provided to users on Alps as containers.
86

9-
!!! cwp
10-
The CPE is not provided on the climage and weather platform
7+
!!! info "CPE is the cray modules"
8+
The familiar modules that provide the `Prgenv-gnu` and `Prgenv-cray` programming environments, and packages like `cray-python` and `cray-fftw`, that will be familiar from the old Piz Daint system.
9+
10+
!!! warning "CPE is not supported on Alps"
11+
The Cray modules were provided on the old Daint system, and CSCS supported their use and provided software built on top of them.
12+
13+
**Alps is a big change - the CPE modules are not provide as officially supported software.**
14+
They are provided _as is_ to users who still need to use them, however CSCS will not be able to provide detailed support for issues that arise when using them.
15+
16+
The recommended method for building and running software is to use [uenv][ref-uenv] or [containers][ref-container-engine].
1117

1218
## CPE in a container
13-
!!! info
14-
Currently it is mandatory to export the EDF path manually
15-
```
19+
20+
[](){#ref-cpe-versions}
21+
### Available versions
22+
23+
The `PrgEnv-gnu` and `PrgEnv-cray` programming environments are provided as separate containers, instead of having both in one container, named `gnu-$version` and `cray-$version` respectively.
24+
The `version` is the CPE version in the container.
25+
26+
| | `zen2` | `gh200` |
27+
|-----------------|----------|---------|
28+
| `cpe-gnu-24.7` | ❌ | ✅ |
29+
| `cpe-cray-24.7` | ❌ | ✅ |
30+
31+
!!! warning "only available on gh200"
32+
The CPE container is only provided on systems with the "[container engine][ref-container-engine]" container runtime, which is currently the Grace-Hopper systems [Daint][ref-cluster-daint], [Clariden][ref-cluster-clariden] and [Santis][ref-cluster-santis].
33+
34+
### How to use
35+
36+
!!! info "Before you start"
37+
To use the CPE containers as documented below, you need to set the `EDF_PATH` environment variable.
38+
```console
1639
export EDF_PATH=/capstor/scratch/cscs/anfink/shared/cpe/edf:$EDF_PATH
1740
```
41+
Note that the current location of the EDF files is temporary, to work around a file system bug.
42+
The environment variable __will not be required__ once this issue is fixed.
1843

19-
To start a container with the cray programming environment, there are pre-defined EDF toml files. Currently they are stored in `/capstor/scratch/cscs/anfink/shared/cpe/edf`, but this will change in the future to a different location.
20-
Every toml file in this edf path is a programming environment that can be started
21-
```
22-
[daint][anfink@daint-ln001 ~]$ ls -lh /capstor/scratch/cscs/anfink/shared/cpe/edf
23-
total 8.0K
24-
-rw-r--r--+ 1 anfink csstaff 175 Apr 23 11:48 cpe-cray-24.07.toml
25-
-rw-r--r--+ 1 anfink csstaff 174 Apr 23 11:32 cpe-gnu-24.07.toml
26-
```
27-
Following the naming scheme from the output, you can spawn a container with
28-
```
29-
srun --environment=cpe-cray-24.07 --pty bash
44+
To start a session with the CPE (see [available-versions][ref-cpe-versions] above):
45+
```console
46+
$ srun --environment=cpe-cray-24.07 --pty bash
3047
```
3148
Once the container starts up you can directly use the programming environment, because there will be modules loaded by default at startup.
32-
```
33-
[daint][anfink@daint-ln001 ~]$ srun -p debug --environment=cpe-gnu-24.07 --pty bash
34-
[daint][anfink@nid005417 /]$ module list
3549

36-
Currently Loaded Modules:
37-
1) craype-arm-grace 2) craype-network-ofi 3) xpmem/2.9.6 4) gcc-native/13.2 5) craype/2.7.32 6) PrgEnv-gnu/8.5.0 7) cray-mpich/8.1.30 8) cuda/12.6 9) craype-accel-nvidia90
50+
```console
51+
$ srun -p debug --environment=cpe-cray-24.07 --pty bash
52+
$ module list
3853

54+
Currently Loaded Modules:
55+
1) craype-arm-grace 4) cce/18.0.0 7) cray-mpich/8.1.30
56+
2) craype-network-ofi 5) craype/2.7.32 8) cuda/12.6
57+
3) xpmem/2.9.6 6) PrgEnv-cray/8.5.0 9) craype-accel-nvidia90
3958

4059

41-
[daint][anfink@nid005417 /]$ module avail
60+
$ module avail
4261

43-
----------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/mpi/gnu/12.0/ofi/1.0/cray-mpich/8.0 -----------------------------------------------------------------------------------------------------------------------------------------------------
62+
---- /opt/cray/pe/lmod/modulefiles/mpi/crayclang/17.0/ofi/1.0/cray-mpich/8.0 ----
4463
cray-hdf5-parallel/1.14.3.1 cray-parallel-netcdf/1.12.3.13
4564

46-
----------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/comnet/gnu/12.0/ofi/1.0 -----------------------------------------------------------------------------------------------------------------------------------------------------------
65+
---------- /opt/cray/pe/lmod/modulefiles/comnet/crayclang/17.0/ofi/1.0 ----------
4766
cray-mpich-abi/8.1.30 cray-mpich/8.1.30 (L)
4867

49-
---------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/mix_compilers ----------------------------------------------------------------------------------------------------------------------------------------------------------------
50-
gcc-native-mixed/13.2
51-
52-
-------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/compiler/gnu/12.0 --------------------------------------------------------------------------------------------------------------------------------------------------------------
68+
------------- /opt/cray/pe/lmod/modulefiles/compiler/crayclang/17.0 -------------
5369
cray-hdf5/1.14.3.1 cray-libsci/24.07.0
5470

55-
-------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/cpu/arm-grace/1.0 --------------------------------------------------------------------------------------------------------------------------------------------------------------
56-
cray-fftw/3.3.10.8
57-
58-
----------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/craype-targets/default ------------------------------------------------------------------------------------------------------------------------------------------------------------
59-
craype-accel-amd-gfx908 craype-accel-amd-gfx942 craype-accel-nvidia80 craype-hugepages128M craype-hugepages256M craype-hugepages32M craype-hugepages64M craype-network-ofi (L) craype-x86-milan-x craype-x86-spr-hbm
60-
craype-accel-amd-gfx90a craype-accel-host craype-accel-nvidia90 (L) craype-hugepages16M craype-hugepages2G craype-hugepages4M craype-hugepages8M craype-network-ucx craype-x86-milan craype-x86-spr
61-
craype-accel-amd-gfx940 craype-accel-nvidia70 craype-arm-grace (L) craype-hugepages1G craype-hugepages2M craype-hugepages512M craype-network-none craype-x86-genoa craype-x86-rome craype-x86-trento
71+
------------------ /opt/cray/pe/lmod/modulefiles/mix_compilers ------------------
72+
cce-mixed/18.0.0
6273

63-
-------------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/pe/lmod/modulefiles/core ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
64-
PrgEnv-gnu/8.5.0 (L) cray-libsci_acc/24.07.0 cray-pmi/6.1.15.19 cray-python/3.11.7 craype/2.7.32 (L) gcc-native/13.2 (L)
74+
---------------- /opt/cray/pe/lmod/modulefiles/cpu/arm-grace/1.0 ----------------
75+
cray-fftw/3.3.10.8
6576

66-
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cray/modulefiles ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
77+
------------- /opt/cray/pe/lmod/modulefiles/craype-targets/default --------------
78+
craype-accel-amd-gfx908 craype-hugepages16M craype-network-none
79+
craype-accel-amd-gfx90a craype-hugepages1G craype-network-ofi (L)
80+
craype-accel-amd-gfx940 craype-hugepages256M craype-network-ucx
81+
craype-accel-amd-gfx942 craype-hugepages2G craype-x86-genoa
82+
craype-accel-host craype-hugepages2M craype-x86-milan-x
83+
craype-accel-nvidia70 craype-hugepages32M craype-x86-milan
84+
craype-accel-nvidia80 craype-hugepages4M craype-x86-rome
85+
craype-accel-nvidia90 (L) craype-hugepages512M craype-x86-spr-hbm
86+
craype-arm-grace (L) craype-hugepages64M craype-x86-spr
87+
craype-hugepages128M craype-hugepages8M craype-x86-trento
88+
89+
---------------------- /opt/cray/pe/lmod/modulefiles/core -----------------------
90+
PrgEnv-cray/8.5.0 (L) cray-libsci_acc/24.07.0 cray-python/3.11.7
91+
cce/18.0.0 (L) cray-pmi/6.1.15.19 craype/2.7.32 (L)
92+
93+
----------------------------- /opt/cray/modulefiles -----------------------------
6794
xpmem/2.9.6 (L)
6895

69-
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- /opt/cscs/modulefiles ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
96+
----------------------------- /opt/cscs/modulefiles -----------------------------
7097
cuda/12.6 (L)
7198

72-
Where:
73-
L: Module is loaded
74-
75-
If the avail list is too long consider trying:
76-
77-
"module --default avail" or "ml -d av" to just list the default modules.
78-
"module overview" or "ml ov" to display the number of modules for each name.
79-
80-
Use "module spider" to find all possible modules and extensions.
81-
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
82-
83-
84-
[daint][anfink@nid005417 /]$ CC --version
85-
g++-13 (SUSE Linux) 13.3.0
86-
Copyright (C) 2023 Free Software Foundation, Inc.
87-
This is free software; see the source for copying conditions. There is NO
88-
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
89-
90-
[daint][anfink@nid005417 /]$
99+
$ CC --version
100+
Cray clang version 18.0.0 (0e4696aa65fa9549bd5e19c216678cc98185b0f7)
101+
Target: aarch64-unknown-linux-gnu
102+
Thread model: posix
103+
InstalledDir: /opt/cray/pe/cce/18.0.0/cce-clang/aarch64/share/../bin
91104
```
92105

93106
The recommended way of using CPE in a container is to start the container, and use `$SCRATCH` and `$STORE` to interact with persistent data. Please remember that any data that is written to a directory that is not mounted from the host system will be lost, after the container stops.
94107

95-
By default, the paths `/capstor`, `/iopsstor` are mounted to the same paths inside the container.
108+
!!! note
109+
By default, the paths `/capstor`, `/iopsstor` are mounted to the same paths inside the container.
96110

97111
Additionally `/users` will be mounted at `/users.host`, so you can access data in your home folder, but with a slightly different path. This is on purpose, and you can override this behaviour by writing your own [EDF file][ref-ce-edf-reference], especially using the key `base_environment`, referencing the predefined CPE environment files and override what you would like to change.

‎docs/index.md‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,6 @@ The Alps Research infrastructure hosts multiple platforms and clusters targeting
4040

4141
[:octicons-arrow-right-24: Alps Storage](alps/storage.md)
4242

43-
</div>
44-
45-
46-
<div class="grid cards" markdown>
47-
4843
- :fontawesome-solid-layer-group: __Accounts and Projects__
4944

5045
The first step is to get an account and a project

‎mkdocs.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ nav:
7171
- build-install/index.md
7272
- 'uenv': build-install/uenv.md
7373
- 'Containers': build-install/containers.md
74-
- 'Cray Programming Environment (CPE)': build-install/cpe.md
74+
- 'Cray modules (CPE))': build-install/cpe.md
7575
- 'Python with pip': build-install/pip.md
7676
- 'Debugging and Performance Analysis':
7777
- software/devtools/index.md

0 commit comments

Comments
 (0)