diff --git a/docs/alps/hardware.md b/docs/alps/hardware.md
index 58173255..83eac33c 100644
--- a/docs/alps/hardware.md
+++ b/docs/alps/hardware.md
@@ -38,22 +38,44 @@ This approach to cooling provides greater efficiency for the rack-level cooling,
Alps was installed in phases, starting with the installation of 1024 AMD Rome dual socket CPU nodes in 2020, through to the main installation of 2,688 Grace-Hopper nodes in 2024.
-There are currently four node types in Alps, with another becoming available in 2025:
+There are currently five node types in Alps:
-| type | blades | nodes | CPU sockets | GPU devices |
-| ---- | ------:| -----:| -----------:| -----------:|
-| NVIDIA GH200 | 1344 | 2688 | 10,752 | 10,752 |
-| AMD Rome | 256 | 1024 | 2,048 | -- |
-| NVIDIA A100 | 72 | 144 | 144 | 576 |
-| AMD MI250x | 12 | 24 | 24 | 96 |
-| AMD MI300A | 64 | 128 | 512 | 512 |
+| type | abbreviation | blades | nodes | CPU sockets | GPU devices |
+| ---- | ------- | ------:| -----:| -----------:| -----------:|
+| NVIDIA GH200 | gh200 | 1344 | 2688 | 10,752 | 10,752 |
+| AMD Rome | zen2 | 256 | 1024 | 2,048 | -- |
+| NVIDIA A100 | a100 | 72 | 144 | 144 | 576 |
+| AMD MI250x | mi200 | 12 | 24 | 24 | 96 |
+| AMD MI300A | mi300 | 64 | 128 | 512 | 512 |
[](){#ref-alps-gh200-node}
### NVIDIA GH200 GPU Nodes
-!!! todo
+!!! under-construction
+ The description of the GH200 nodes is a work in progress.
+ We will add more detailed information soon.
+ Please [get in touch](https://github.com/eth-cscs/cscs-docs/issues) if there is information that you want to see here.
+
+There are 24 cabinets, in 4 rows with 6 cabinets per row, and each cabinet contains 112 nodes (for a total of 448 GH200):
+* 8 chassis per cabinet
+* 7 blades per chassis
+* 2 nodes per blade
+
+!!! info "Why 7 blades per chassis?"
+ A chassis can contain up to 8 blades, however Alps' gh200 chassis are underpopulated so that we can increase the amount of power delivered to each GPU.
-Blanca Peak
+Each node contains four Grace-Hopper modules and four corresponding network interface cards (NICS) per blade, as illustrated below:
+
+
+
+??? info "Node xname"
+ There are two boards per blade with one node per board.
+ This is different to the `zen2` CPU-only nodes (used for example in Eiger) that have two nodes per board for a total of four nodes per blade.
+ As such, there are no `n1` nodes in the xname list, e.g.:
+ ```
+ x1100c0s6b0n0
+ x1100c0s6b1n0
+ ```
[](){#ref-alps-zen2-node}
### AMD Rome CPU Nodes
@@ -79,6 +101,8 @@ Bard Peak
[](){#ref-alps-mi300-node}
### AMD MI300A GPU Nodes
+
+
!!! todo
Parry Peak
diff --git a/docs/contributing/index.md b/docs/contributing/index.md
index e9728925..0e7ea43e 100644
--- a/docs/contributing/index.md
+++ b/docs/contributing/index.md
@@ -229,6 +229,66 @@ They stand out better from the main text, and can be collapsed by default if nee
If an admonition is collapsed by default, it should have a title.
+We provide some custom admonitions.
+
+#### Change
+
+For adding information about a change, originally designed for recording updates to clusters.
+
+=== "Rendered"
+ !!! change "2025-04-17"
+ * SLURM was upgraded to version 25.1.
+ * uenv was upgraded to v0.8
+
+ Old changes can be folded:
+
+ ??? change "2025-02-04"
+ * The new Scratch cleanup policy was implemented
+ * NVIDIA driver was updated
+
+=== "Markdown"
+ ```
+ !!! change "2025-04-17"
+ * SLURM was upgraded to version 25.1.
+ * uenv was upgraded to v0.8
+ ```
+
+ Old changes can be folded:
+
+ ```
+ ??? change "2025-02-04"
+ * The new Scratch cleanup policy was implemented
+ * NVIDIA driver was updated
+ ```
+
+#### Under construction
+
+For marking incomplete sections.
+
+=== "Rendered"
+ !!! under-construction
+ This is not finished yet!
+
+=== "Markdown"
+ ```
+ !!! under-construction
+ This is not finished yet!
+ ```
+
+#### Todo
+
+As a placeholder for documentation that needs to be written.
+
+=== "Rendered"
+ !!! todo
+ Add some common error messages and how to fix them.
+
+=== "Markdown"
+ ```
+ !!! todo
+ Add some common error messages and how to fix them.
+ ```
+
### Code blocks
Use [code blocks](https://squidfunk.github.io/mkdocs-material/reference/code-blocks/) when you want to display monospace text in a programming language, terminal output, configuration files etc.
diff --git a/docs/images/alps/gh200-schematic.svg b/docs/images/alps/gh200-schematic.svg
new file mode 100644
index 00000000..de981f66
--- /dev/null
+++ b/docs/images/alps/gh200-schematic.svg
@@ -0,0 +1,2676 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GPU HBM3
+
+
+
+
+ GPU HBM3
+
+
+
+
+ ~94 GB Numa 4(-11 MIG)
+
+
+
+
+ Hopper
+
+
+
+
+ GPU
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ≤4000GB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GPU HBM3
+
+
+
+
+ GPU HBM3
+
+
+
+
+ ~94 GB Numa 20(-27 MIG)
+
+
+
+
+ Hopper
+
+
+
+
+ GPU
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ≤4000GB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GPU HBM3
+
+
+
+
+ GPU HBM3
+
+
+
+
+ ~94 GB Numa 28(-35 MIG)
+
+
+
+
+ Hopper
+
+
+
+
+ GPU
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ≤4000GB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ NVIDIA G H200 Grace Hopper Superchip
+
+
+
+
+ NVIDIA G H200 Grace Hopper Superchip
+
+
+
+
+ NVIDIA G H200 Grace Hopper Superchip
+
+
+
+
+ NVIDIA G H200 Grace Hopper Superchip
+
+
+
+
+
+
+ High-Speed
+
+
+ IO
+
+
+ High-Speed
+
+
+ IO
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Grace
+
+
+ CPU
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ≤500GB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ~117 GB Numa 0
+
+
+
+
+
+
+ CPU LPDDR5X
+
+
+
+
+ CPU LPDDR5X
+
+
+
+
+
+
+
+ Grace
+
+
+ CPU
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ≤500GB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ~120 GB Numa1
+
+
+
+
+
+
+ CPU LPDDR5X
+
+
+
+
+ CPU LPDDR5X
+
+
+
+
+
+
+ Hardware Coherency
+
+
+
+
+
+
+
+ Grace
+
+
+ CPU
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ≤500GB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ~120 GB Numa 2
+
+
+
+
+
+
+ CPU LPDDR5X
+
+
+
+
+ CPU LPDDR5X
+
+
+
+
+
+
+
+ Grace
+
+
+ CPU
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ≤500GB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ~120 GB Numa 3
+
+
+
+
+
+
+ CPU LPDDR5X
+
+
+
+
+ CPU LPDDR5X
+
+
+
+
+
+
+
+
+
+ 512GB/s
+
+
+
+
+ 512GB/s
+
+
+
+
+ 16x PCIe-5
+
+
+
+
+ 16x PCIe-5
+
+
+
+
+ 4x
+
+
+
+
+ 4x
+
+
+
+
+
+
+ High-Speed
+
+
+ IO
+
+
+ High-Speed
+
+
+ IO
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 512GB/s
+
+
+
+
+ 512GB/s
+
+
+
+
+ 16x PCIe-5
+
+
+
+
+ 16x PCIe-5
+
+
+ 4x
+
+
+ 4x
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ NVLink-C2C
+
+
+ 900GB/s
+
+
+ NVLink-C2C
+
+
+ 900GB/s
+
+
+ NVLink-C2C
+
+
+ 900GB/s
+
+
+
+
+
+
+
+
+
+
+ NVLink-C2C
+ 450GB/s per Dir
+
+
+ 900GB/s Tot
+
+
+ NVLink-C2C
+ 450GB/s per Dir
+
+
+ 900GB/s Tot
+
+
+ NVLink-C2C
+ 450GB/s per Dir
+
+
+ 900GB/s Tot
+
+
+ NVLink-C2C
+ 450GB/s per Dir
+
+
+ 900GB/s Tot
+
+
+
+
+
+ GPU Al l2all: 3x6
+
+
+ P2P N VLink
+
+
+ 150GB/s
+ per direction
+
+
+
+
+
+
+ CPU Al l2all
+
+
+ Grace Interconnect
+
+
+ ~75GB/s
+ per direction
+
+
+
+
+
+
+ Slingshot hsn0
+
+
+ 200Gb/s (25GB/s)
+
+
+
+
+ Slingshot hsn3
+
+
+ 200Gb/s (25GB/s)
+
+
+
+
+ Slingshot hsn1
+
+
+ 200Gb/s (25GB/s)
+
+
+
+
+
+
+ Slingshot
+ hsn2
+
+
+ 200Gb/s (25GB/s)
+
+
+
+
+
+
+ ~378 GB (GPU)
+
+
+
+
+ ~477 GB (CPU)
+
+
+
+
+
+
+
+
+ = ~855 GB Unified memory
+
+
+
+
+ 4xGH200 NVIDIA Grace Hopper Superch ips
+
+
+
+
+ +
+
+
+
+
+
+ CSCS ALPS HPE Cray EX GH200 Node
+
+
+
+
+
+
+
+
+
+
+ Slingshot Switches
+ connecting to other AL PS
+ nodes, internet,
+
+
+ storage (SSD/HDD)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GPU HBM3
+
+
+
+
+ GPU HBM3
+
+
+
+
+ Hopper
+
+
+
+
+ GPU
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ≤4000GB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ~94 GB Numa 12(-19 MIG)
+
+
+
+
+
+
diff --git a/docs/images/alps/mi300-schematic.svg b/docs/images/alps/mi300-schematic.svg
new file mode 100644
index 00000000..665e8016
--- /dev/null
+++ b/docs/images/alps/mi300-schematic.svg
@@ -0,0 +1,4790 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 6x38 = 228 GPU
+
+
+ Compute Units
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Unified HBM3 Memory
+
+
+
+
+
+
+
+
+ Unified HBM3 Memory
+
+
+ 3x8 = 24 AM D Zen4
+
+
+ x86 CPU cores
+
+
+
+
+ ~125 GiB Numa0
+
+
+
+
+
+
+
+
+
+
+
+ ≤5300GiB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 6x38 = 228 GPU
+
+
+ Compute Units
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Unified HBM3 Memory
+
+
+
+
+
+
+
+
+ Unified HBM3 Memory
+
+
+ 3x8 = 24 AM D Zen4
+
+
+ x86 CPU cores
+
+
+
+
+ ~125 GiB Numa2
+
+
+
+
+
+
+
+
+
+
+
+ ≤5300GiB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 6x38 = 228 GPU
+
+
+ Compute Units
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Unified HBM3 Memory
+
+
+
+
+
+
+
+
+ Unified HBM3 Memory
+
+
+ 3x8 = 24 AM D Zen4
+
+
+ x86 CPU cores
+
+
+
+
+ ~125 GiB Numa1
+
+
+
+
+
+
+
+
+
+
+
+ ≤5300GiB/s Tot
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 6x38 = 228 GPU
+
+
+ Compute Units
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Unified HBM3 Memory
+
+
+
+
+
+
+
+
+ Unified HBM3 Memory
+
+
+ 3x8 = 24 AM D Zen4
+
+
+ x86 CPU cores
+
+
+
+
+ ~125 GiB Numa3
+
+
+
+
+
+
+
+
+
+
+
+ ≤5300GiB/s Tot
+
+
+
+
+
+
+
+ AMD INSTINCT™ MI300A AP U
+
+
+
+
+ AMD INSTINCT™ MI300A AP U
+
+
+
+
+ AMD INSTINCT™ MI300A AP U
+
+
+
+
+ AMD INSTINCT™ MI300A AP U
+
+
+
+
+
+
+ High-Speed
+
+
+ IO
+
+
+ High-Speed
+
+
+ IO
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Hardware Coherency
+
+
+
+
+
+
+
+
+ 512GiB/s
+
+
+
+
+ 512GiB/s
+
+
+
+
+ 16x PCIe-5
+
+
+
+
+ 16x PCIe-5
+
+
+
+
+ 4x
+
+
+
+
+ 4x
+
+
+
+
+
+
+ High-Speed
+
+
+ IO
+
+
+ High-Speed
+
+
+ IO
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 512GiB/s
+
+
+
+
+ 512GiB/s
+
+
+
+
+ 16x PCIe-5
+
+
+
+
+ 16x PCIe-5
+
+
+ 4x
+
+
+ 4x
+
+
+
+
+
+ Slingshot hsn2
+
+
+ 200Gb/s (25GiB/s)
+
+
+
+
+ Slingshot hsn3
+
+
+ 200Gb/s (25GiB/s)
+
+
+
+
+ Slingshot hsn0
+
+
+ 200Gb/s (25GiB/s)
+
+
+
+
+
+
+ Slingshot
+ hsn1
+
+
+ 200Gb/s (25GiB/s)
+
+
+
+
+
+
+
+
+
+
+ ~477 GiB Unified HBM3 Memory
+
+
+
+
+ 4xAMD INSTINCT™ MI300A APU
+
+
+
+
+
+ CSCS ALPS HPE Cray EX MI300A Node
+
+
+
+
+
+
+
+
+
+
+
+
+ Slingshot Switches
+ connecting to other AL PS
+ nodes, internet,
+
+
+ storage (SSD/HDD)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 6 Infinity Fabric™
+
+
+ Links
+
+
+ providing a total of
+ 256GB/s per
+ direction between
+ each AP U
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css
index 4a9eec90..0e9ae3c6 100644
--- a/docs/stylesheets/extra.css
+++ b/docs/stylesheets/extra.css
@@ -53,6 +53,22 @@
mask-image: var(--md-admonition-icon--alps);
}
+/* under-construction admonition */
+.md-typeset .admonition.under-construction,
+.md-typeset details.under-construction {
+ border-color: rgb(0, 0, 0);
+}
+.md-typeset .under-construction > .admonition-title,
+.md-typeset .under-construction > summary {
+ background-color: rgba(255, 255, 0, 0.3);
+}
+.md-typeset .under-construction > .admonition-title::before,
+.md-typeset .under-construction > summary::before {
+ background-color: rgb(0, 0, 0);
+ -webkit-mask-image: var(--md-admonition-icon--digging);
+ mask-image: var(--md-admonition-icon--digging);
+}
+
.md-nav__item .md-nav__link--active {
font-weight: bold;
}