Merge branch 'main' into kubernetes-docs

Author: bcumming

.github/CODEOWNERS

@@ -4,6 +4,7 @@ docs/services/firecrest @jpdorsch @ekouts
 docs/services/kubernetes @eliaoggian
 docs/software/communication @Madeeks @msimberg
 docs/software/devtools/linaro @jgphpc
+docs/software/devtools/vihps @jgphpc
 docs/software/prgenv/linalg.md @finkandreas @msimberg
 docs/software/sciapps/cp2k.md @abussy @RMeli
 docs/software/sciapps/lammps.md @nickjbrowning

.github/actions/spelling/allow.txt

@@ -347,3 +347,8 @@ xname
 xpmem
 youtube
 zstd
+HPS
+jobscript
+Scalasca
+tracefile
+Vampir

docs/guides/terminal.md

@@ -8,26 +8,24 @@ This documentation is a collection of guides, hints, and tips for setting up you
 
 Every user has a shell that will be used when they log in, with [bash](https://www.gnu.org/software/bash/) as the default shell for new users at CSCS.
 
-At CSCS the vast majority of users stick with the default `bash`: at the time of writing, of over 1000 users on Daint, over 99% were using bash.
-
 !!! example "Which shell am I using?"
 
     Run the following command after logging in:
 
     ```console
-    $ getent passwd | grep $USER
-    bcumming:*:22008:1000:Benjamin Cumming, CSCS:/users/bcumming:/usr/local/bin/bash
+    $ echo $SHELL
+    /usr/local/bin/bash
     ```
 
-    The last entry in the output points to the shell of the user, in this case `/usr/local/bin/bash`.
-
 !!! tip
     If you would like to change your shell, for example to [zsh](https://www.zsh.org), you have to open a [service desk](https://jira.cscs.ch/plugins/servlet/desk) ticket to request the change. You can't make the change yourself.
 
 
 !!! warning
-    Because `bash` is used by all CSCS staff and the overwhelming majority of users, it is the best tested, and safest default.
-
+    If you are comfortable with another shell (like Zsh or Fish), you are welcome to switch.
+    Just keep in mind that some tools and instructions might not work the same way outside of `bash`.
+    Since our support and documentation are based on the default setup, using a different shell might make it harder to follow along or get help.
+    
     We strongly recommend against using cshell - tools like uenv are not tested against it.
 
 [](){#ref-guides-terminal-arch}
@@ -76,3 +74,49 @@ export PATH=$xdgbase/bin:$PATH
 !!! note "XDG what?"
     The [XDG base directory specification](https://specifications.freedesktop.org/basedir-spec/latest/) is used by most applications to determine where to look for configurations, and where to store data and temporary files.
 
+[](){#ref-guides-terminal-bashrc}
+## Modifying bashrc
+
+The `~/.bashrc` in your home directory is executed __every time__ you log in, and there is no way to log in without executing it.
+
+It is strongly recommended that customization in `~/.bashrc` should be kept to the bare minimum:
+
+1. It sets a fixed set of environment options every time you log in, and all downstream scripts and Slurm batch jobs might assume that these commands have run, so that later modifications to `~/.bashrc` can break workflows in ways that are difficult to debug.
+    * If a script or batch job requires environment modifications, implement them there.
+    * In other words, move the definition of environment used by a workflow to the workflow definition.
+1. It makes it difficult for CSCS to provide support, because it is difficult for support staff to reproduce your environment, and it can take a lot of back and forth before we determine that the root cause of an issue is a command in `~/.bashrc`.
+
+
+!!! warning "Do not call `module` in bashrc"
+    Calls to `module use` and `module load` in `~/.bashrc` is possible, however avoid it for the reasons above.
+    If there are module commands in your `~/.bashrc`, remember to provide a full copy of `~/.bashrc` with support tickets.
+
+!!! danger "Do not call `uenv` in bashrc"
+    The `uenv` command is designed for creating isolated environments, and calling it in `~/.bashrc` will not work as expected.
+    See the [uenv docs][ref-uenv-customenv] for more information about how to create bespoke uenv environments that can be started with a single command.
+
+??? note "Help, I broke bashrc!"
+    It is possible to add commands to bashrc that will stop you from being able to log in.
+    The author of these docs has done it more than once, after ignoring their own advice.
+
+    For example, if the command `exit` is added to `~/.bashrc` you will be logged out every time you log in.
+
+    The first thing to try is to execute a command that will back up `~/.bashrc`, and remove `~/.bashrc`:
+    ```bash
+    ssh eiger.cscs.ch 'bash --norc --noprofile -c "mv ~/.bashrc ~/.bashrc.back"'
+    ```
+    If this works, you can then log in normally, and edit the backup and copy it back to `~/.bashrc`.
+
+    If there is a critical error, like calling `exit`, the approach above won't work.
+    In such cases, the only solution that doesn't require root permissions is to log in and hit `<ctrl-c>` during the log in.
+    With luck, this will cancel the login process before `~/.bashrc` is executed, and you will be able to edit and fix `~/.bashrc`.
+    Note that you might have to try a few times to get the timing right.
+
+    If this does not work, create a [service desk ticket][ref-get-in-touch] with the following message:
+
+    !!! example "Help request"
+        My bashrc has been modified, and I can't log in any longer to `insert-system-name`.
+        My username is `insert-cscs-username`.
+        Can you please make a backup copy of my bashrc, i.e. `mv ~/.bashrc ~/.bashrc.back`,
+        so that I can log in and fix the issue.
+

docs/images/devtools/vihps/sphexa_cube.png

@@ -216,19 +216,21 @@ It is the same thing.
 [](){#ref-cicd-pipeline-triggers-api}
 #### API call triggering
 - It is possible to trigger a pipeline via an API call
-- Create a file named `data.yaml`, with the content
-```yaml
-ref: main
-pipeline: pipeline_name
-variables:
-  MY_VARIABLE: some_value
-  ANOTHER_VAR: other_value
-```
-Send a POST request to the middleware
-```bash
-curl -X POST -u 'repository_id:webhook_secret' --data-binary @data.yaml https://cicd-ext-mw.cscs.ch/ci/pipeline/trigger
-```
-- replace repository_id and webhook_secret with your repository id and the webhook secret.
+- Create a file `data.yaml`, with the content
+    ```yaml title="data.yaml"
+    ref: main
+    pipeline: pipeline_name
+    variables:
+      MY_VARIABLE: some_value
+      ANOTHER_VAR: other_value
+    ```
+- Send a POST request to the middleware (replace `repository_id` and `webhook_secret`)
+    ```console
+    $ curl -X POST -u 'repository_id:webhook_secret' --data-binary @data.yaml https://cicd-ext-mw.cscs.ch/ci/pipeline/trigger
+    ```
+- To trigger a pull-request use `ref: 'pr:<pr-number>'`
+- To trigger a tag use `ref: 'tag:<tag-name>'`
+- To trigger on a specific commit SHA use `ref: 'sha:<commit-sha>'`
 
 ### Understanding the underlying workflow
 Typical users do not need to know the underlying workflow behind the scenes, so you can stop reading here.

docs/images/devtools/vihps/sphexa_vampir.png

@@ -15,7 +15,7 @@ There are three ways to do so:
     $ srun --environment=./.edf/ubuntu.toml echo "Hello"    # from ${HOME}. 
     ```
 
- 3. **From EDF search paths**: the name of EDF in the [EDF search path][ref-ce-edf-search-path]. `--environment` also accepts the EDF filename without the `.toml` extension:
+ 3. **From EDF search paths**: the name of EDF in the [EDF search path][ref-ce-edf-search-path]. Notice that in this way, `--environment` accepts the EDF filename **without** the `.toml` extension.
 
     ```console 
     $ srun --environment=ubuntu echo "Hello" 

docs/services/cicd.md

@@ -28,6 +28,6 @@ This ensures that computational resources are utilized to their fullest potentia
 Learning to analyze the performance of an applications effectively is crucial to build a deeper understanding of how your code interacts with the underlying hardware.
 In this section we introduce the various performance analysis solutions available at CSCS.
 
-* [Linaro Forge MAP][ref-devtools-map]
 * [NVIDIA Nsight Developer Tools][ref-devtools-nsight]
-
+* [Linaro Forge MAP][ref-devtools-map]
+* [VI-HPS Tools][ref-devtools-vihps]

docs/software/container-engine/run.md

@@ -0,0 +1,136 @@
+[](){#ref-devtools-vihps}
+# VI-HPS tools
+
+The [VI-HPS](https://www.vi-hps.org/tools) Institute (Virtual Institute for High Productivity Supercomputing) provides tools that can assist developers of simulation codes to address their needs in performance analysis.
+
+## [Score-P](https://www.vi-hps.org/projects/score-p/overview/overview.html)
+
+[Score-P](https://www.vi-hps.org/projects/score-p/overview/overview.html)
+is a highly scalable instrumentation and measurement infrastructure for profiling, event tracing, and online analysis. It supports a wide range of HPC platforms and programming models. Score-P provides core measurement services for a range of specialized analysis tools, such as Vampir, Scalasca and others.
+
+## [Vampir](https://www.vi-hps.org/tools/vampir.html) 
+
+[Vampir](https://www.vi-hps.org/tools/vampir.html) 
+is a performance visualizer that allows to quickly study the program runtime behavior at a fine level of details. This includes the display of detailed performance event recordings over time in timelines and aggregated profiles. Interactive navigation and zooming are the key features of the tool, which help to quickly identify inefficient or faulty parts of a program.
+
+!!! info
+    While Score-P does not require a license, [Vampir](https://vampir.eu/licensing) does. CSCS standard license allows to read trace files with up to 256 concurrent threads of execution.
+
+    The Vampir GUI is currently available only on `x86-64` CPU based systems and is not provided via a uenv (more details in the Quickstart guide below).
+    You can use Score-P to generate OTF2 traces files on Alps compute nodes and then visualize the results with Vampir on a x86-64 CPU based system (for instance Eiger, LUMI or using your own license). 
+
+## [Cube and Scalasca](http://www.vi-hps.org/tools/scalasca.html)
+
+[Cube and Scalasca](http://www.vi-hps.org/tools/scalasca.html)
+support the performance optimization of parallel programs with a collection of scalable trace-based tools for in-depth analyses of concurrent behavior. The analysis identifies potential performance bottlenecks - in particular those concerning communication and synchronization - and offers guidance in exploring their causes.
+
+## Quickstart guide
+
+The VI-HPS uenv is named `scorep` and it can be loaded into your environment as explained here and in the [uenv documentation][ref-uenv].
+
+!!! example "Finding and pulling available `scorep` versions"
+
+    ```console
+    uenv image find scorep
+    # uenv                 arch   system  id                 size(MB) date
+    # scorep/9.2-gcc12:v1  gh200  daint   bfd3b46d30404f2c   7,602    2025-07-14
+    # scorep/9.2-gcc13:v1  gh200  daint   3c0357a490c81f32   7,642    2025-07-14
+
+    uenv image pull scorep/9.2-gcc13:v1
+    # pulling 3c0357a490c81f32 100.00%
+    ```    
+
+    This uenv is configured to be mounted in the `/user-environment` path.
+
+!!! example "Start the `scorep` uenv"
+
+    ```bash
+    uenv start scorep/9.2-gcc13:v1 -v default
+    
+    uenv status # (1)!
+    scorep --version # 9.2
+    scalasca --version # 2.6.2
+    cubelib-config --version # 4.9
+    otf2-print --version # 3.1.1
+    
+    find /user-environment/ -name scorep.pdf # (2)!
+    ```
+    
+    1. Test that everything has been mounted correctly and that the tools are in the PATH
+    2. A PDF version of the user guide is available in the uenv.
+
+!!! example "Recompile your code with `scorep` on Alps"
+
+    ```bash
+    # Building with CMake requires the following steps
+    ## Invoke cmake with the scorep wrapper disabled:
+    SCOREP_WRAPPER=OFF \
+        cmake -S src -B build \
+        -DCMAKE_CXX_COMPILER=scorep-mpic++ \
+        -DCMAKE_C_COMPILER=scorep-mpicc \
+        -DCMAKE_CUDA_COMPILER=scorep-nvcc \
+        -DCMAKE_CUDA_ARCHITECTURES=90 # [...]
+
+    ## Then build with the scorep wrapper enabled:
+    SCOREP_WRAPPER=ON \
+        cmake --build build
+    ```
+
+!!! example "Run your application with `scorep` on Alps"
+
+    Pick one of the report type in your jobscript before running the executable compiled with `scorep`:
+    
+    === "Profiling"
+
+        - Profiling gives an overview of the performance of your simulation on Alps
+    
+        ```bash
+        export SCOREP_ENABLE_PROFILING=true
+        # Call-path profiling: CUBE4 data format (profile.cubex)
+        ```
+
+        - Then run your job as usual with `srun` or `sbatch` on Alps,
+        - Copy the generated profile `profile.cubex` to your laptop,
+        - Install the [Cube](https://www.scalasca.org/scalasca/software) tool on your laptop,
+        - Analyze the results with the GUI:
+            - performance metric (left panel)
+            - call path (middle panel)
+            - system resource (right panel)
+
+        ```bash
+        /Applications/Cube/4.9/Cube.app/Contents/MacOS/maccubegui.sh \
+        ./profile.cubex
+        ```    
+
+        ![sphexa_cube](../../images/devtools/vihps/sphexa_cube.png){ width="90%"}
+    
+    === "Tracing"
+
+        - Tracing allows a detailed analysis of the performance of your simulation on Alps
+    
+        ```bash
+        export SCOREP_ENABLE_TRACING=true
+        # Event-based tracing: OTF2 data format (traces.otf2)
+        ```
+
+        - Then run your job as usual with `srun` or `sbatch` on Alps,
+        - Analyze the results with the GUI:
+
+        ```bash
+        ssh -X eiger.cscs.ch  # Vampir GUI requires x86_64 ⚠️
+        /capstor/store/cscs/userlab/vampir/10.6.1/bin/vampir \
+        ./traces.otf2
+        ```
+
+        !!! info
+            - Tracing allows more detailed analysis but will also make your simulation run longer than with profiling,
+            - `scorep-score` allows to estimate the size of an OTF2 tracefile from a CUBE profile,
+              it can also help to reduce the overhead of tracing via filtering:
+            ```bash
+            scorep-score -g profile.cubex # generate filter file
+            scorep-score -f initial_scorep.filter profile.cubex
+            export SCOREP_FILTERING_FILE='initial_scorep.filter'
+            ```
+            - The [user guide](https://perftools.pages.jsc.fz-juelich.de/cicd/scorep/tags/scorep-9.2/html/group__SCOREP__User.html#gaab4b3ccc2b169320c1d3bf7fe19165f9) provides more details about how to reduce overhead.
+
+        ![sphexa_vampir](../../images/devtools/vihps/sphexa_vampir.png){ width="90%"}

docs/software/devtools/index.md

@@ -15,6 +15,9 @@ transition state optimization using NEB or dimer method. See [CP2K Features] for
     [CP2K] is provided on [ALPS][platforms-on-alps] via [uenv][ref-uenv].
     Please have a look at the [uenv documentation][ref-uenv] for more information about uenvs and how to use them.
 
+!!! warning "Known issues"
+    Please check CP2K's [known issues](#known-issues) and whether they are relevant to your work. They may impact your calculations in subtle ways, potentially leading to a waste of resources.
+
 ??? note "Changelog"
 
     ??? note "2025.1"
@@ -422,6 +425,20 @@ See [manual.cp2k.org/CMake] for more details.
 
 ## Known issues
 
+### Older uenv versions on Eiger
+
+After the migration to Eiger.Alps, calculations relying on older uenv versions (`2024.1:v1`, `2024.2:v1`, `2024.3:v1`) sometimes crash unexpectedly with a segmentation fault. The problem has been identify as coming from the `libxsmm` library, used as a backend in DBCSR. To avoid this issue, it is recommended to upgrade to a newer uenv.
+
+In case a specific `2024.x` version of CP2K is required, crashes can be avoided by switching to the `BLAS` backend of DBCSR. This can be done by adding the following in the `&GLOBAL` subsection of the input file:
+
+```bash
+&GLOBAL
+    &DBCSR
+        MM_DRIVER BLAS
+    &END DBCSR
+&END GLOBAL
+```
+
 ### DLA-Future
 
 The `cp2k/2025.1:v2` uenv provides CP2K with [DLA-Future] support enabled, in the `cp2k-dlaf` view.