CE: Added pages with guidelines for images on Alps

Author: Madeeks

docs/software/container-engine/examples/guidelines-images.md

@@ -0,0 +1,35 @@
+[](){#ref-ce-guidelines-images}
+# Guidelines for images on Alps
+
+This section offers some guidelines about creating and using container images that achieve good performance on the Alps reseach infrastructure.
+The section focuses on foundational components (such as communication libraries) which are essential to enabling performant effective usage of Alps' capabilities, rather than full application use cases.
+Synthetic benchmarks are also used to showcase quantitative performance.
+
+!!! important
+    The Containerfiles and examples provided in this section are intended to serve as general reference and starting point.
+    They are not meant to represent all possible combinations and versions of software capable of running efficiently on Alps.
+
+    In the same vein, please note that the content presented here is not intended to represent images officially supported by CSCS staff.
+
+Below is a summary of the software suggested and demonstrated throughout this section:
+
+- Base components:
+    - CUDA 12.8.1
+    - GDRCopy 2.5.1
+    - Libfabric 1.22.0
+    - UCX 1.19.0
+- MPI implementations
+    - MPICH 4.3.1
+    - OpenMPI 5.0.8
+- Other programming libraries
+    - NVSHMEM 3.4.5
+- Synthetic benchmarks
+    - OSU Micro-benchmarks 7.5.1
+    - NCCL Tests 2.17.1
+
+The content is organized in pages which detail container images building incrementally upon each other:
+
+- a base image installing baseline libraries and frameworks (e.g. CUDA, libfabric)
+- MPI implementations (MPICH, OpenMPI)
+- NVSHMEM
+- NCCL tests

docs/software/container-engine/examples/image-comm-fwk.md

@@ -0,0 +1,105 @@
+[](){#ref-ce-guidelines-images-commfwk}
+# Communication frameworks image
+
+This page describes a container image providing foundational software components for achieving efficient execution on Alps nodes with NVIDIA GPUs.
+
+The most important aspect to consider for performance of containerized applications is related to use of high-speed networks,
+therefore this image mainly installs communication frameworks and libraries, besides general utility tools.
+In particular, the [Libfabric](https://ofiwg.github.io/libfabric/) framework (also known as Open Fabrics Interfaces - OFI) is required to interface applications with the Slingshot high-speed network.
+
+At runtime, the container engine [CXI hook][ref-ce-cxi-hook] will replace the Libfabric libraries inside the container with the corresponding libraries on the host system.
+This will ensure access to the Slingshot interconnect.
+
+This image is not intended to be used on its own, but to serve as a base to build higher-level software (e.g. MPI implementations) and application stacks.
+For this reason, no performance results are provided in this page.
+
+A build of this image is currently hosted on the [Quay.io](https://quay.io/) registry at the following reference:
+`quay.io/ethcscs/comm-fwk:ofi1.22-ucx1.19-cuda12.8`.
+The image name `comm-fwk` is a shortened form of "communication frameworks".
+
+## Contents
+
+- Ubuntu 24.04
+- CUDA 12.8.1
+- GDRCopy 2.5.1
+- Libfabric 1.22.0
+- UCX 1.19.0
+
+## Containerfile
+```Dockerfile
+ARG ubuntu_version=24.04
+ARG cuda_version=12.8.1
+FROM docker.io/nvidia/cuda:${cuda_version}-cudnn-devel-ubuntu${ubuntu_version}
+
+RUN apt-get update \
+    && DEBIAN_FRONTEND=noninteractive \
+       apt-get install -y \
+        build-essential \
+        ca-certificates \
+        pkg-config \
+        automake \
+        autoconf \
+        libtool \
+        cmake \
+        gdb \
+        strace \
+        wget \
+        git \
+        bzip2 \
+        python3 \
+        gfortran \
+        rdma-core \
+        numactl \
+        libconfig-dev \
+        libuv1-dev \
+        libfuse-dev \
+        libfuse3-dev \
+        libyaml-dev \
+        libnl-3-dev \
+        libnuma-dev \
+        libsensors-dev \
+        libcurl4-openssl-dev \
+        libjson-c-dev \
+        libibverbs-dev \
+        --no-install-recommends \
+    && rm -rf /var/lib/apt/lists/*
+
+ARG gdrcopy_version=2.5.1
+RUN git clone --depth 1 --branch v${gdrcopy_version} https://github.com/NVIDIA/gdrcopy.git \
+    && cd gdrcopy \
+    && export CUDA_PATH=${CUDA_HOME:-$(echo $(which nvcc) | grep -o '.*cuda')} \
+    && make CC=gcc CUDA=$CUDA_PATH lib \
+    && make lib_install \
+    && cd ../ && rm -rf gdrcopy
+
+# Install libfabric
+ARG libfabric_version=1.22.0
+RUN git clone --branch v${libfabric_version} --depth 1 https://github.com/ofiwg/libfabric.git \
+    && cd libfabric \
+    && ./autogen.sh \
+    && ./configure --prefix=/usr --with-cuda=/usr/local/cuda --enable-cuda-dlopen --enable-gdrcopy-dlopen --enable-efa \
+    && make -j$(nproc) \
+    && make install \
+    && ldconfig \
+    && cd .. \
+    && rm -rf libfabric
+
+# Install UCX
+ARG UCX_VERSION=1.19.0
+RUN wget https://github.com/openucx/ucx/releases/download/v${UCX_VERSION}/ucx-${UCX_VERSION}.tar.gz \
+    && tar xzf ucx-${UCX_VERSION}.tar.gz \
+    && cd ucx-${UCX_VERSION} \
+    && mkdir build \
+    && cd build \
+    && ../configure --prefix=/usr --with-cuda=/usr/local/cuda --with-gdrcopy=/usr/local --enable-mt --enable-devel-headers \
+    && make -j$(nproc) \
+    && make install \
+    && cd ../.. \
+    && rm -rf ucx-${UCX_VERSION}.tar.gz ucx-${UCX_VERSION}
+```
+
+## Notes
+- The image is based on an official NVIDIA CUDA image, and therefore already provides the NCCL library, alongside a complete CUDA installation.
+- Communication frameworks are built with explicit support for CUDA and GDRCopy.
+- The libfabric EFA provider is included to leave open the possibility to experiment with derived images on AWS infrastructure as well.
+- Although only the libfabric framework is required to support Alps' Slingshot network, this image also packages the UCX communication framework to allow building a broader set of software (e.g. some OpenSHMEM implementations) and supporting optimized Infiniband communication as well.