Skip to content

Commit 1550417

Browse files
authored
Merge branch 'main' into spellcheck
2 parents 3fc44d9 + 6eed060 commit 1550417

File tree

6 files changed

+288
-31
lines changed

6 files changed

+288
-31
lines changed

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,5 @@ docs/software/communication @Madeeks @msimberg
44
docs/software/devtools/linaro @jgphpc
55
docs/software/prgenv/linalg.md @finkandreas @msimberg
66
docs/software/sciapps/cp2k.md @abussy @RMeli
7+
docs/software/sciapps/gromacs.md @kanduri
78
docs/software/ml @boeschf

docs/build-install/cpe.md

Lines changed: 107 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,111 @@
11
# Cray Programming Environment (CPE)
22

3-
!!! warning
4-
you don't want to use this
3+
The Cray Programming Environment (CPE) is a suite of software: programming environments, compilers, libraries and tools.
54

6-
!!! mlp
7-
The CPE is not provided on the machine learning platform.
5+
The CPE is provided to users on Alps as containers.
86

9-
!!! cwp
10-
The CPE is not provided on the climate and weather platform
7+
!!! info "CPE is the cray modules"
8+
The familiar modules that provide the `Prgenv-gnu` and `Prgenv-cray` programming environments, and packages like `cray-python` and `cray-fftw`, that will be familiar from the old Piz Daint system.
9+
10+
!!! warning "CPE is not supported on Alps"
11+
The Cray modules were provided on the old Daint system, and CSCS supported their use and provided software built on top of them.
12+
13+
**Alps is a big change - the CPE modules are not provide as officially supported software.**
14+
They are provided _as is_ to users who still need to use them, however CSCS will not be able to provide detailed support for issues that arise when using them.
15+
16+
The recommended method for building and running software is to use [uenv][ref-uenv] or [containers][ref-container-engine].
17+
18+
## CPE in a container
19+
20+
[](){#ref-cpe-versions}
21+
### Available versions
22+
23+
The `PrgEnv-gnu` and `PrgEnv-cray` programming environments are provided as separate containers, instead of having both in one container, named `gnu-$version` and `cray-$version` respectively.
24+
The `version` is the CPE version in the container.
25+
26+
| | `zen2` | `gh200` |
27+
|-----------------|----------|---------|
28+
| `cpe-gnu-24.7` |||
29+
| `cpe-cray-24.7` |||
30+
31+
!!! warning "only available on gh200"
32+
The CPE container is only provided on systems with the "[container engine][ref-container-engine]" container runtime, which is currently the Grace-Hopper systems [Daint][ref-cluster-daint], [Clariden][ref-cluster-clariden] and [Santis][ref-cluster-santis].
33+
34+
### How to use
35+
36+
!!! info "Before you start"
37+
To use the CPE containers as documented below, you need to set the `EDF_PATH` environment variable.
38+
```console
39+
export EDF_PATH=/capstor/scratch/cscs/anfink/shared/cpe/edf:$EDF_PATH
40+
```
41+
Note that the current location of the EDF files is temporary, to work around a file system bug.
42+
The environment variable __will not be required__ once this issue is fixed.
43+
44+
To start a session with the CPE (see [available-versions][ref-cpe-versions] above):
45+
```console
46+
$ srun --environment=cpe-cray-24.07 --pty bash
47+
```
48+
Once the container starts up you can directly use the programming environment, because there will be modules loaded by default at startup.
49+
50+
```console
51+
$ srun -p debug --environment=cpe-cray-24.07 --pty bash
52+
$ module list
53+
54+
Currently Loaded Modules:
55+
1) craype-arm-grace 4) cce/18.0.0 7) cray-mpich/8.1.30
56+
2) craype-network-ofi 5) craype/2.7.32 8) cuda/12.6
57+
3) xpmem/2.9.6 6) PrgEnv-cray/8.5.0 9) craype-accel-nvidia90
58+
59+
60+
$ module avail
61+
62+
---- /opt/cray/pe/lmod/modulefiles/mpi/crayclang/17.0/ofi/1.0/cray-mpich/8.0 ----
63+
cray-hdf5-parallel/1.14.3.1 cray-parallel-netcdf/1.12.3.13
64+
65+
---------- /opt/cray/pe/lmod/modulefiles/comnet/crayclang/17.0/ofi/1.0 ----------
66+
cray-mpich-abi/8.1.30 cray-mpich/8.1.30 (L)
67+
68+
------------- /opt/cray/pe/lmod/modulefiles/compiler/crayclang/17.0 -------------
69+
cray-hdf5/1.14.3.1 cray-libsci/24.07.0
70+
71+
------------------ /opt/cray/pe/lmod/modulefiles/mix_compilers ------------------
72+
cce-mixed/18.0.0
73+
74+
---------------- /opt/cray/pe/lmod/modulefiles/cpu/arm-grace/1.0 ----------------
75+
cray-fftw/3.3.10.8
76+
77+
------------- /opt/cray/pe/lmod/modulefiles/craype-targets/default --------------
78+
craype-accel-amd-gfx908 craype-hugepages16M craype-network-none
79+
craype-accel-amd-gfx90a craype-hugepages1G craype-network-ofi (L)
80+
craype-accel-amd-gfx940 craype-hugepages256M craype-network-ucx
81+
craype-accel-amd-gfx942 craype-hugepages2G craype-x86-genoa
82+
craype-accel-host craype-hugepages2M craype-x86-milan-x
83+
craype-accel-nvidia70 craype-hugepages32M craype-x86-milan
84+
craype-accel-nvidia80 craype-hugepages4M craype-x86-rome
85+
craype-accel-nvidia90 (L) craype-hugepages512M craype-x86-spr-hbm
86+
craype-arm-grace (L) craype-hugepages64M craype-x86-spr
87+
craype-hugepages128M craype-hugepages8M craype-x86-trento
88+
89+
---------------------- /opt/cray/pe/lmod/modulefiles/core -----------------------
90+
PrgEnv-cray/8.5.0 (L) cray-libsci_acc/24.07.0 cray-python/3.11.7
91+
cce/18.0.0 (L) cray-pmi/6.1.15.19 craype/2.7.32 (L)
92+
93+
----------------------------- /opt/cray/modulefiles -----------------------------
94+
xpmem/2.9.6 (L)
95+
96+
----------------------------- /opt/cscs/modulefiles -----------------------------
97+
cuda/12.6 (L)
98+
99+
$ CC --version
100+
Cray clang version 18.0.0 (0e4696aa65fa9549bd5e19c216678cc98185b0f7)
101+
Target: aarch64-unknown-linux-gnu
102+
Thread model: posix
103+
InstalledDir: /opt/cray/pe/cce/18.0.0/cce-clang/aarch64/share/../bin
104+
```
105+
106+
The recommended way of using CPE in a container is to start the container, and use `$SCRATCH` and `$STORE` to interact with persistent data. Please remember that any data that is written to a directory that is not mounted from the host system will be lost, after the container stops.
107+
108+
!!! note
109+
By default, the paths `/capstor`, `/iopsstor` are mounted to the same paths inside the container.
110+
111+
Additionally `/users` will be mounted at `/users.host`, so you can access data in your home folder, but with a slightly different path. This is on purpose, and you can override this behaviour by writing your own [EDF file][ref-ce-edf-reference], especially using the key `base_environment`, referencing the predefined CPE environment files and override what you would like to change.

docs/index.md

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,6 @@ The Alps Research infrastructure hosts multiple platforms and clusters targeting
4040

4141
[:octicons-arrow-right-24: Alps Storage](alps/storage.md)
4242

43-
</div>
44-
45-
46-
<div class="grid cards" markdown>
47-
4843
- :fontawesome-solid-layer-group: __Accounts and Projects__
4944

5045
The first step is to get an account and a project

docs/services/firecrest.md

Lines changed: 14 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33

44
FirecREST is a RESTful API for programmatically accessing High-Performance Computing resources, developed at CSCS.
55

6-
Users can make use of FirecREST to automate access to HPC, enabling [CI/CD pipelines](https://github.com/eth-cscs/firecrest/tree/master/examples/CI-pipeline), [workflow managers](https://github.com/eth-cscs/firecrest/tree/master/examples/airflow-operators), and other tools against HPC resources.
6+
Users can make use of FirecREST to automate access to HPC, enabling [CI/CD pipelines](https://eth-cscs.github.io/firecrest-v2/use_cases/CI-pipeline/), [workflow managers](https://github.com/eth-cscs/firecrest/tree/master/examples/airflow-operators), and other tools against HPC resources.
77

8-
Additionally, scientific platform developers can integrate FirecREST into [web-enabled portals](https://firecrest-ui-hpc.cscs.ch) and [applications](https://github.com/eth-cscs/firecrest/tree/master/examples/UI-client-credentials), allowing them to securely access authenticated and authorized CSCS services such as job submission and data transfer on HPC systems.
8+
Additionally, scientific platform developers can integrate FirecREST into [web-enabled portals](https://my.hpcp.cscs.ch) and [applications](https://github.com/eth-cscs/firecrest/tree/master/examples/UI-client-credentials), allowing them to securely access authenticated and authorized CSCS services such as job submission and data transfer on HPC systems.
99

1010
Users can make HTTP requests to perform the following operations:
1111

@@ -17,18 +17,17 @@ Users can make HTTP requests to perform the following operations:
1717

1818
Starting early 2025, CSCS has introduced a new version of the API: [FirecREST version 2](https://eth-cscs.github.io/firecrest-v2).
1919

20-
Version 2 is faster, easier to use, and more efficient in resource management than its predecessor, therefore, if you are new to FirecREST start directly using **version 2** (if available for your platform).
20+
Version 2 is faster, easier to use, and more efficient in resource management than its predecessor, therefore, if you are new to FirecREST start directly using **version 2**.
2121

22-
If you're using **version 1**, we recommend you to port your applications to use the new version.
22+
!!! warning "Deprecation notice"
23+
If you're using **version 1**, we recommend you to port your applications to use the new version.
24+
We will communicate soon the exact date of the decomission of version 1 (not before Quarter 4 of 2025).
2325

2426
=== "Version 2"
2527

26-
!!!Warning
27-
Documentation for version 2 is still work in progress
28-
2928
For a full feature set, have a look at the latest [FirecREST version 2 API specification](https://eth-cscs.github.io/firecrest-v2/openapi) deployed at CSCS.
3029

31-
Please refer to the [FirecREST-v2 documentation](https://eth-cscs.github.io/firecrest-v2/use) for detailed documentation.
30+
Please refer to the [FirecREST-v2 documentation](https://eth-cscs.github.io/firecrest-v2/user_guide/) for detailed documentation.
3231

3332
=== "Version 1"
3433

@@ -136,7 +135,7 @@ Back on the Subscription Management page, you can review your active subscriptio
136135

137136
![FirecREST subscription management](../images/firecrest/f7t-api-subscriptions-management.png)
138137

139-
To use your Application to access FirecREST, follow the [API documentation](https://firecrest-docs.v1.svc.cscs.ch/).
138+
To use your Application to access FirecREST, follow the [API documentation](https://eth-cscs.github.io/firecrest-v2/openapi).
140139

141140
## Getting Started
142141

@@ -444,14 +443,12 @@ A staging area is used for external transfers and downloading/uploading a file f
444443

445444
## Further Information
446445

447-
* [FirecREST UI for HPC Platform](https://firecrest-ui-hpc.cscs.ch)
448-
* [FirecREST OpenAPI Specification Version 1](https://firecrest.readthedocs.io/en/latest/reference.html)
449-
* [FirecREST OpenAPI Specification Version 2](https://eth-cscs.github.io/firecrest-v2/openapi)
450-
* [FirecREST Docs Version 1](https://firecrest.readthedocs.io/)
451-
* [FirecREST Docs Version 2](https://eth-cscs.github.io/firecrest-v2)
452-
* [Documentation of pyFirecREST (v1/v2)](https://pyfirecrest.readthedocs.io/)
453-
* [FirecREST repository Version 1](https://github.com/eth-cscs/firecrest)
454-
* [FirecREST repository Version 2](https://github.com/eth-cscs/firecrest-v2)
446+
* [FirecREST UI for HPC Platform](https://my.hpcp.cscs.ch)
447+
* [FirecREST UI for ML Platform](https://my.mlp.cscs.ch)
448+
* [FirecREST OpenAPI Specification](https://eth-cscs.github.io/firecrest-v2/openapi)
449+
* [FirecREST Official Docs](https://eth-cscs.github.io/firecrest-v2)
450+
* [Documentation of pyFirecREST](https://pyfirecrest.readthedocs.io/)
451+
* [FirecREST repository](https://github.com/eth-cscs/firecrest-v2)
455452
* [What are JSON Web Tokens](https://jwt.io/introduction)
456453
* [Python Requests](https://requests.readthedocs.io/en/master/user/quickstart)
457454
* [Python Async API Calls](https://docs.aiohttp.org/en/stable/)

0 commit comments

Comments
 (0)