Merge branch 'main' into expand-communication

Author: msimberg

.github/CODEOWNERS

@@ -1,5 +1,6 @@
 * @bcumming @msimberg @RMeli
 docs/services/firecrest @jpdorsch @ekouts
 docs/software/communication @Madeeks @msimberg
+docs/software/devtools/linaro @jgphpc
 docs/software/prgenv/linalg.md @finkandreas @msimberg
 docs/software/sciapps/cp2k.md @abussy @RMeli

docs/access/mfa.md

@@ -50,8 +50,8 @@ Before starting, ensure that the following pre-requisites are satisfied
 
 !!! warning
     If you try to SSH to CSCS systems without setting up MFA, you will be prompted with permission denied error, for example:
-    ```
-    > ssh ela.cscs.ch
+    ```console
+    $ ssh ela.cscs.ch
     [email protected]: Permission denied (publickey).
     Connection closed by UNKNOWN port 65535
     ```

docs/clusters/clariden.md

@@ -84,8 +84,8 @@ See the SLURM documentation for instructions on how to run jobs on the [Grace-Ho
 
 ??? example "how to check the number of nodes on the system"
     You can check the size of the system by running the following command in the terminal:
-    ```terminal
-    > sinfo --format "| %20R | %10D | %10s | %10l | %10A |"
+    ```console
+    $ sinfo --format "| %20R | %10D | %10s | %10l | %10A |"
     | PARTITION            | NODES      | JOB_SIZE   | TIMELIMIT  | NODES(A/I) |
     | debug                | 32         | 1-2        | 30:00      | 3/29       |
     | normal               | 1266       | 1-infinite | 1-00:00:00 | 812/371    |

docs/clusters/santis.md

@@ -98,8 +98,8 @@ See the SLURM documentation for instructions on how to run jobs on the [Grace-Ho
 
 ??? example "how to check the number of nodes on the system"
     You can check the size of the system by running the following command in the terminal:
-    ```terminal
-    > sinfo --format "| %20R | %10D | %10s | %10l | %10A |"
+    ```console
+    $ sinfo --format "| %20R | %10D | %10s | %10l | %10A |"
     | PARTITION            | NODES      | JOB_SIZE   | TIMELIMIT  | NODES(A/I) |
     | debug                | 32         | 1-2        | 30:00      | 3/29       |
     | normal               | 1266       | 1-infinite | 1-00:00:00 | 812/371    |

docs/contributing/index.md

@@ -13,20 +13,20 @@ We use the GitHub fork and pull request model for development:
 Clone your fork repository on your PC/laptop:
 ```bash
 # clone your fork of the repository
-> git clone [email protected]:${githubusername}/cscs-docs.git
-> cd cscs-docs
-> git switch -c 'fix/ssh-alias'
+git clone [email protected]:${githubusername}/cscs-docs.git
+cd cscs-docs
+git switch -c 'fix/ssh-alias'
 # ... make your edits ...
 # add and commit your changes
-> git add <files>
-> git commit -m 'update the ssh docs with aliases for all user lab vclusters'
-> git push origin 'fix/ssh-alias'
+git add <files>
+git commit -m 'update the ssh docs with aliases for all user lab vclusters'
+git push origin 'fix/ssh-alias'
 ```
 Then navigate to GitHub, and create a pull request.
 
 The `serve` script in the root path of the repository can be used to view the docs locally:`
-```
-> ./serve
+```bash
+./serve
 ...
 INFO    -  [08:33:34] Serving on http://127.0.0.1:8000/
 ```
@@ -228,3 +228,56 @@ They stand out better from the main text, and can be collapsed by default if nee
     This note is collapsed, because it uses `???`.
 
 If an admonition is collapsed by default, it should have a title.
+
+### Code blocks
+
+Use [code blocks](https://squidfunk.github.io/mkdocs-material/reference/code-blocks/) when you want to display monospace text in a programming language, terminal output, configuration files etc.
+The documentation uses [pygments](https://pygments.org) for highlighting.
+See [list of available lexers](https://pygments.org/docs/lexers/#) for the languages that you can use for code blocks.
+
+Use [`console`](https://pygments.org/docs/lexers/#pygments.lexers.shell.BashSessionLexer) for interactive sessions with prompt-output pairs:
+
+=== "Markdown"
+
+    ````markdown
+    ```console title="Hello, world!"
+    $ echo "Hello, world!"
+    Hello, world!
+    ```
+    ````
+
+=== "Rendered"
+
+    ```console title="Hello, world!"
+    $ echo "Hello, world!"
+    Hello, world!
+    ```
+
+!!! warning
+    `terminal` is not a valid lexer, but MkDocs or pygments will not warn about using it as a language.
+    The text will be rendered without highlighting.
+
+!!! warning
+    Use `$` as the prompt character, optionally preceded by text.
+    `>` as the prompt character will not be highlighted correctly.
+
+Note the use of `title=...`, which will give the code block a heading.
+
+!!! tip
+    Include a title whenever possible to describe what the code block does or is.
+
+If you want to display commands without output that can easily be copied, use `bash` as the language:
+
+=== "Markdown"
+
+    ````markdown
+    ```bash title="Hello, world!"
+    echo "Hello, world!"
+    ```
+    ````
+
+=== "Rendered"
+
+    ```bash title="Hello, world!"
+    echo "Hello, world!"
+    ```

docs/guides/gb2025.md

@@ -1,7 +1,7 @@
 [](){#ref-gb2025}
 # Gordon Bell and HPL runs 2025
 
-For Gordon Bell and HPL runs in March-April 2025, CSCS has created a reservation on Santis with 1333 nodes (12 cabinets).
+For Gordon Bell and HPL runs in March-April 2025, CSCS has expanded Santis to 1333 nodes (12 cabinets).
 
 For the runs, CSCS has applied some updates and changes that aim to improve performance and scaling scale, particularly for NCCL.
 If you are already familiar with running on Daint, you might have to make some small changes to your current job scripts and parameters, which will be documented here.
@@ -27,6 +27,18 @@ Host santis
 
 The `normal` partition is used with no reservation, which means that that jobs can be submittied without `--partition` and `--reservation` flags.
 
+Timeline:
+
+1. Friday 4th April:
+    * HPE finish HPL runs at 10:30am
+    * CSCS performs testing on the reconfigured system for ~1 hour on the `GB_TESTING_2` reservation
+    * The reservation is removed and all GB teams have access to test and tune applications.
+2. Monday 7th April:
+    * at 4pm the runs will start for the first team
+
+!!! note
+    There will be no special reservation during the open testing and tuning between Friday and Monday.
+
 ### Storage
 
 Your data sets from Daint are available on Santis
@@ -37,51 +49,34 @@ Your data sets from Daint are available on Santis
 
 ## Low Noise Mode
 
-Low noise mode (LNM) is now enabled.
-This confines system processes and operations to the first core of each of the four NUMA regions in a node (i.e., cores 0, 72, 144, 216).
+!!! note
+    Low noise mode has been relaxed, so the previous requirement that you set `OMP_PLACES` and `OMP_PROC_BIND` no longer applies.
+    One core per module is still reserved for system processes.
+    
+Santis uses low noise mode, which reserves one core per Grace-Hopper module (i.e. per 72 cores) for system processes.
+This mode is intended to reduce performance variability caused by system processes interfering with application threads and processes.
+This means that SLURM job scripts must be updated to account for the reserved cores.
 
-The consequence of this setting is that only 71 cores per socket can be requested by an application (for a total of 284 cores instead of 288 cores per node).
+### SLURM
 
-!!! warning "Unable to allocate resources: Requested node configuration is not availabl"
+!!! warning "Unable to allocate resources: Requested node configuration is not available"
     If you try to use all 72 cores on each socket, SLURM will give a hard error, because only 71 are available:
 
-    ```
+    ```console
     # try to run 4 ranks per node, with 72 cores each
-    > srun -n4 -N1 -c72 --reservation=reshuffling ./build/affinity.mpi
+    $ srun -n4 -N1 -c72 ./build/affinity.mpi
     srun: error: Unable to allocate resources: Requested node configuration is not available
     ```
 
-One consequence of this change is that thread affinity and OpenMP settings that worked on Daint might cause large slowdown in the new configuration.
-
-### SLURM
-
 Explicitly set the number of cores per task using the `--cpus-per-task/-c` flag, e.g.:
 ```
 #SBATCH --cpus-per-task=64
-#SBATCH --cpus-per-task=71
 ```
 or
 ```
 srun -N1 -n4 -c71 ...
 ```
 
-**Do not** use the `--cpu-bind` flag to control affinity
-
-* this can cause large slowdown, particularly with `--cpu-bind=socket`. We are investigating how to fix this.
-
-If you see significant slowdown and you want to report it, please provide the output of using the `--cpu-bind=verbose` flag.
-
-### OpenMP
-
-If your application uses OpenMP, try setting the following in your job script:
-
-```bash
-export OMP_PLACES=cores
-export OMP_PROC_BIND=close
-```
-
-Without these settings, we have observed application slowdown due to poor thread placement.
-
 ## NCCL
 
 !!! todo

docs/guides/terminal.md

@@ -14,7 +14,7 @@ At CSCS the vast majority of users stick with the default `bash`: at the time of
 
     Run the following command after logging in:
 
-    ```terminal
+    ```console
     $ getent passwd | grep $USER
     bcumming:*:22008:1000:Benjamin Cumming, CSCS:/users/bcumming:/usr/local/bin/bash
     ```

docs/software/container-engine.md

@@ -50,13 +50,13 @@ Since the `ubuntu.toml` file is located in the [EDF search path][ref-ce-edf-sear
     The above terminal snippet demonstrates how to launch a containerized environment using Slurm with the `--environment` option.
     Click on the :fontawesome-solid-circle-plus: icon for information on each command.
 
-    ```bash
-    daint-ln002 > srun --environment=ubuntu --pty bash   # (1)
+    ```console
+    [daint-ln002]$ srun --environment=ubuntu --pty bash   # (1)
 
-    nid005333 > pwd                                    # (2)
+    [nid005333]$ pwd                                    # (2)
     /capstor/scratch/cscs/<username>
 
-    nid005333 > cat /etc/os-release                    # (3)
+    [nid005333]$ cat /etc/os-release                    # (3)
     PRETTY_NAME="Ubuntu 24.04 LTS"
     NAME="Ubuntu"
     VERSION_ID="24.04"
@@ -71,8 +71,8 @@ Since the `ubuntu.toml` file is located in the [EDF search path][ref-ce-edf-sear
     UBUNTU_CODENAME=noble
     LOGO=ubuntu-logo
 
-    nid005333 > exit                                  # (4)
-    daint-ln002 >
+    [nid005333]$ exit                                  # (4)
+    [daint-ln002]$
     ```
 
     1.  Starting an interactive shell session within the Ubuntu 24.04 container deployed on a compute node using `srun --environment=ubuntu --pty bash`.

docs/software/devtools/index.md

@@ -1,12 +1,12 @@
 [](){#ref-software-devtools}
 # Debugging and Performance Analysis tools
 
-Debugging and Performance Analysis tools can assist users in developing and optimizing scientific parallel applications, especially in a high-performance computing (HPC) environment.
-Efficient tools can significantly improve workflows and save valuable computational resources.
+Debugging and performance analysis tools can assist users in developing and optimizing scientific parallel applications, especially in a high-performance computing (HPC) environment.
+These tools can significantly improve workflows and save valuable computational resources.
 
-CSCS provides debuggers and performance analysis tools on Alps Clusters.
+CSCS provides debuggers and performance analysis tools on [Alps][ref-alps] clusters.
 
-!!! note "get in touch"
+!!! note "Get in touch"
     If you have issues or questions about debugging or performance analysis tools, please do not hesitate to [contact us][ref-get-in-touch].
 
 [](){#ref-devtools-debug}

docs/software/devtools/linaro-ddt.md

@@ -2,12 +2,12 @@
 # Linaro DDT
 
 DDT allows source-level debugging of Fortran, C, C++ and Python codes.
-It can be used for debugging serial, multi-threaded (OpenMP), multi-process (MPI) and accelerated (CUDA, OpenACC) programs running on research and production systems, including the CSCS Alps system.
+It can be used for debugging serial, multi-threaded (OpenMP), multi-process (MPI), and accelerated (CUDA, OpenACC) programs running on research and production systems, including the CSCS [Alps][ref-alps] system.
 DDT can be executed either with its graphical user interface or from the command-line.
 
 !!! note
-    Linaro DDT is provided in the `linaro-forge` uenv.
-    Before using DDT, please read the [`linaro-forge` documentation][ref-uenv-linaro], which explains how to download and set up the latest version and set it up.
+    Linaro DDT is provided in the `linaro-forge` [uenv][ref-uenv].
+    Before using DDT, please read the [`linaro-forge` uenv documentation][ref-uenv-linaro], which explains how to download and set up the latest version.
 
 ## User guide
 
@@ -18,7 +18,7 @@ The following guide will walk through the steps required to build and debug an a
 Once the uenv is loaded and activated, the program to debug must be compiled with the `-g` (for CPU) and `-G` (for GPU) debugging flags.
 For example, we can build a CUDA test with a user environment:
 
-```terminal
+```bash
 uenv start prgenv-gnu:24.11:v1 --view=default
 nvcc -c -arch=sm_90 -g -G test_gpu.cu
 mpicxx -g test_cpu.cpp test_gpu.o -o myexe
@@ -27,29 +27,33 @@ mpicxx -g test_cpu.cpp test_gpu.o -o myexe
 ### Launch Linaro DDT
 
 To use the DDT client with uenv, it must be launched in `Manual Launch` mode
-(assuming that it is connected to Alps via `Remote Launch`):
+(assuming that it is connected to [Alps][ref-alps] via `Remote Launch`):
 
-=== "on local machine"
+=== "On local machine"
 
     Start DDT, and connect to the target cluster using the drop down menu for `Remote Launch`.
+    If you don't have a target cluster,
+    the [`linaro-forge` uenv documentation][ref-uenv-linaro] explains how to set up the connection the first time.
 
-    Click on `Manual launch`, set the number of processes to listen to, then wait for the slurm job to start (see the "on Alps" tab).
+    Click on `Manual launch`, set the number of processes to listen to, then wait for the Slurm job to start 
+    (see the "on Alps" tab for how to start the Slurm job).
 
     <img src="https://raw.githubusercontent.com/jgphpc/cornerstone-octree/ddt/scripts/img/ddt/0.png" width="600" />
 
-=== "on Alps"
+=== "On Alps"
 
     Log into the system and launch with the `srun` command:
 
-    ```terminal
-    # start a session with both the PE used to build your application
-    # and the linaro-forge uenv mounted
-    > uenv start prgenv-gnu/24.11:v1,linaro-forge/24.1.1:v1 --view=prgenv-gnu:default
-    > source /user-tools/activate
-
-    > srun -N1 -n4 -t15 -pdebug ./cuda_visible_devices.sh   ddt-client   ./myexe
+    ```console
+    $ uenv start prgenv-gnu/24.11:v1,linaro-forge/24.1.1:v1 --view=prgenv-gnu:default # (1)!
+    $ source /user-tools/activate
+    $ srun -N1 -n4 -t15 -pdebug ./cuda_visible_devices.sh   ddt-client   ./myexe
     ```
 
+    1. Start a session with both the uenv used to build your application and the `linaro-forge` uenv mounted.
+
+
+
 ### Start debugging
 
 By default, DDT will pause execution on the call to `MPI_Init`:
@@ -65,14 +69,16 @@ There are two mechanisms for controlling program execution:
 
 === "Stop at"
 
-    Execution can be paused in every CUDA kernel launch by activating the default breakpoints from the Control menu:
+    Execution can be paused in every CUDA kernel launch by activating the default breakpoints from the `Control` menu:
 
     <img src="https://raw.githubusercontent.com/jgphpc/cornerstone-octree/ddt/scripts/img/ddt/4.png" width="400" />
 
 
-This screenshot shows a debugging session on 128 gpus:
+??? example  "Debugging with 128 GPUs"
+    
+    This screenshot shows a debugging session on 128 GPUs:
 
-![DDTgpus](https://raw.githubusercontent.com/jgphpc/cornerstone-octree/ddt/scripts/img/ddt/5.png)
+    ![DDTgpus](https://raw.githubusercontent.com/jgphpc/cornerstone-octree/ddt/scripts/img/ddt/5.png)
 
 More informations regarding how to use Linaro DDT are provided in the Forge [User Guide](https://docs.linaroforge.com/latest/html/forge/index.html).