update publi

slaizet · slaizet · commit c5387eeac91d · 2023-05-31T16:28:26.000+01:00
diff --git a/docs/source/pages/installation.rst b/docs/source/pages/installation.rst
@@ -2,61 +2,67 @@
 Installation
 ============
 
-Bulding and installing 2decomp-fft
-==================================
-
 Building
 ===========
 
 The build system is driven by `cmake`. It is good practice to directly point to the MPI Fortran wrapper that you would like to use to guarantee consistency between Fortran compiler and MPI. This can be done by setting the default Fortran environmental variable 
-```
-export FC=my_mpif90
-```
+
+``export FC=my_mpif90``
+
 To generate the build system run
-```
-cmake -S $path_to_sources -B $path_to_build_directory -DOPTION1 -DOPTION2 ...
-```
-If the directory does not exist it will be generated and it will contain the configuration files. The configuration can be further
-edited by using the `ccmake` utility as
-```
-ccmake $path_to_build_directory
-```
-and editing as desired, variables that are likely of interest are: `CMAKE_BUILD_TYPE` and `FFT_Choice`;
-additional variables can be shown by entering "advanced mode" by pressing `t`.
-By default a `RELEASE` build will built, other options for `CMAKE_BUILD_TYPE` are `DEBUG` and `DEV` which
-turn on debugging flags and additionally try to catch coding errors at compile time, respectively.
+
+``cmake -S $path_to_sources -B $path_to_build_directory -DOPTION1 -DOPTION2 ...``
+
+If the directory does not exist it will be generated and it will contain the configuration files. The configuration can be further edited by using the `ccmake` utility as
+
+``ccmake $path_to_build_directory``
+
+and editing as desired, variables that are likely of interest are: `CMAKE_BUILD_TYPE` and `FFT_Choice`; additional variables can be shown by entering "advanced mode" by pressing `t`.
+
+By default a `RELEASE` build will built, other options for `CMAKE_BUILD_TYPE` are `DEBUG` and `DEV` which turn on debugging flags and additionally try to catch coding errors at compile time, respectively.
+
 The behaviour of debug and development versions of the library can be changed before the
-initialization using the variable ``decomp_debug`` or the environment variable ``DECOMP_2D_DEBUG``.
-The value provided with the environment variable must be a positive integer below 9999.
+initialization using the variable ``decomp_debug`` or the environment variable ``DECOMP_2D_DEBUG``. The value provided with the environment variable must be a positive integer below 9999.
 
 Two `BUILD_TARGETS` are available namely `mpi` and `gpu`.  For the `mpi` target no additional options should be required. whereas for `gpu` extra options are necessary at the configure stage. Please see section [GPU Compilation](#gpu-compilation)
 
 Once the build system has been configured, you can build `2decomp&fft` by running
-```
-cmake --build $path_to_build_directory -j <nproc>
-```
+
+``cmake --build $path_to_build_directory -j <nproc>``
+
 appending `-v` will display additional information about the build, such as compiler flags.
 
 After building the library can be tested. Please see section [Testing and examples](#testing-and-examples)
 
 Options can be added to change the level of verbosity. Finally, the build library can be installed by running 
-```
-cmake --install $path_to_build_directory
-```
-The default location for `libdecomp2d.a` is `$path_to_build_directory/opt/lib`or  `$path_to_build_directory/opt/lib64` unless the variable `CMAKE_INSTALL_PREFIX` is modified.
+
+``cmake --install $path_to_build_directory``
+
+The default location for *libdecomp2d.a* is
+
+``$path_to_build_directory/opt/lib`` or
+
+``$path_to_build_directory/opt/lib64``
+
+unless the variable *CMAKE_INSTALL_PREFIX* is modified.
+
 The module files generated by the build process will similarly be installed to `$path_to_build_directory/opt/install`, users of the library should add this to the include paths for their program.
 
 As indicated above, by default a static `libdecomp2d.a` will be compiled, if desired a shared library can be built by setting `BUILD_SHARED_LIBS=ON` either on the command line:
-```
-cmake -S $path_to_sources -B $path_to_build_directory -DBUILD_SHARED_LIBS=ON
-```
+
+::
+   
+     cmake -S $path_to_sources -B $path_to_build_directory -DBUILD_SHARED_LIBS=ON
+
 or by editing the configuration using `ccmake`.
+
 This might be useful for a centralised install supporting multiple users that is upgraded over time.
 
 Occasionally a clean build is required, this can be performed by running
-```
-cmake --build $path_to_build_directory --target clean
-```
+
+::
+   
+     cmake --build $path_to_build_directory --target clean
 
 GPU compilation
 ===============
@@ -65,22 +71,23 @@ The library can perform multi GPU offoloading using the NVHPC compiler suite for
 The implementation is based on CUDA-aware MPI and NVIDIA Collective Communication Library (NCCL).
 The FFT is based on cuFFT. 
 
-To properly configure for GPU build the following needs to be used 
-```
-cmake -S $path_to_sources -B $path_to_build_directory -DBUILD_TARGET=gpu
-```
+To properly configure for GPU build the following needs to be used
+
+``cmake -S $path_to_sources -B $path_to_build_directory -DBUILD_TARGET=gpu``
+
 Note, further configuration can be performed using `ccmake`, however the initial configuration of GPU builds must include the `-DBUILD_TARGET=gpu` flag as shown above.
 
 By default CUDA aware MPI will be used together with `cuFFT` for the FFT library. The configure will automatically look for the GPU architecture available on the system. If you are building on a HPC system please use a computing node for the installation. Useful variables to be added are 
 
  - `-DENABLE_NCCL=yes` to activate the NCCL collectives
  - `-DENABLE_MANAGED=yes` to activate the automatic memory management form the NVHPC compiler
-If you are getting the following error
-```
--- The CUDA compiler identification is unknown  
+
+   If you are getting the following error
+
+``-- The CUDA compiler identification is unknown  
 CMake Error at /usr/share/cmake/Modules/CMakeDetermineCUDACompiler.cmake:633 (message):  
-Failed to detect a default CUDA architecture. 
-```
+Failed to detect a default CUDA architecture.``,
+
 It is possible that your default C compiler is too recent and not supported by `nvcc` . You might be able to solve the issue by adding 
  - `-DCMAKE_CUDA_HOST_COMPILER=$supported_gcc`
  
@@ -89,75 +96,78 @@ It is possible that your default C compiler is too recent and not supported by `
 Linking from external codes
 ==============================
 
-Codes using Makefiles
-=========================
+**Codes using Makefiles**
+
 
 When building a code that links 2decomp-fft using a Makefile you will need to add the include and link paths as appropriate (`inlude/` and `lib/` under the installation directory, respectively).
-```
-DECOMP_ROOT = /path/to/2decomp-fft
-DECOMP_BUILD_DIR = $(DECOMP_ROOT)/build
-DECOMP_INSTALL_DIR ?= $(DECOMP_BUILD_DIR)/opt # Use default unless set by user
 
-INC += -I$(DECOMP_INSTALL_DIR)/include
+::
+   
+     DECOMP_ROOT = /path/to/2decomp-fft
+     DECOMP_BUILD_DIR = $(DECOMP_ROOT)/build
+     DECOMP_INSTALL_DIR ?= $(DECOMP_BUILD_DIR)/opt # Use default unless set by user
 
-# Users build/link targets
-LIBS = -L$(DECOMP_INSTALL_DIR)/lib64 -L$(DECOMP_INSTALL_DIR)/lib -ldecomp2d
+     INC += -I$(DECOMP_INSTALL_DIR)/include
 
-OBJ = my_exec.o
+     # Users build/link targets
+     LIBS = -L$(DECOMP_INSTALL_DIR)/lib64 -L$(DECOMP_INSTALL_DIR)/lib -ldecomp2d
 
-my_exec: $(OBJ)
-	$(F90) -o $@ $(OBJ) $(LIBS)
+     OBJ = my_exec.o
 
-```
+     my_exec: $(OBJ)
+	$(F90) -o $@ $(OBJ) $(LIBS)
+	
 In case 2decomp-fft has been compiled with an external FFT, such as FFTW3, `LIBS` should also contain the following 
-```
-FFTW3_PATH=/my_path_to_FFTW/lib
-LIBFFT=-L$(FFTW3_PATH) -lfftw3 -lfftw3f
-LIBS += $(LIBFFT)
-```
-In case of 2decomp-fft compiled for GPU with NVHPC, linking against cuFFT is mandatory 
-```
-LIBS += -cudalib=cufft
-```
-In case of NCCL the following is required 
-```
-LIBS += -cudalib=cufft,nccl 
-```
-It is also possible to drive the build and installation of 2decomp-fft from a Makefile such as in the following example code
-```
-FC = mpif90
-BUILD = Release
 
-DECOMP_ROOT = /path/to/2decomp-fft
-DECOMP_BUILD_DIR = $(DECOMP_ROOT)/build
-DECOMP_INSTALL_DIR ?= $(DECOMP_BUILD_DIR)/opt # Use default unless set by user
+::
+   
+     FFTW3_PATH=/my_path_to_FFTW/lib
+     LIBFFT=-L$(FFTW3_PATH) -lfftw3 -lfftw3f
+     LIBS += $(LIBFFT)
 
-INC += -I$(DECOMP_INSTALL_DIR)/include
+In case of 2decomp-fft compiled for GPU with NVHPC, linking against cuFFT is mandatory
 
-# Users build/link targets
-LIBS = -L$(DECOMP_INSTALL_DIR)/lib64 -L$(DECOMP_INSTALL_DIR)/lib -ldecomp2d
+``LIBS += -cudalib=cufft``
 
-# Building libdecomp.a
-$(DECOMP_INSTALL_DIR)/lib/libdecomp.a:
-	FC=$(FC) cmake -S $(DECOMP_ROOT) -B $(DECOMP_BUILD_DIR) -DCMAKE_BUILD_TYPE=$(BUILD) -DCMAKE_INSTALL_PREFIX=$(DECOMP_INSTALL_DIR)
-	cmake --build $(DECOMP_BUILD_DIR) --target decomp2d
-	cmake --build $(DECOMP_BUILD_DIR) --target install
+In case of NCCL the following is required 
 
-# Clean libdecomp.a
-clean-decomp:
-	cmake --build $(DECOMP_BUILD_DIR) --target clean
-	rm -f $(DECOMP_INSTALL_DIR)/lib/libdecomp.a
-```
+``LIBS += -cudalib=cufft,nccl``
 
+It is also possible to drive the build and installation of 2decomp-fft from a Makefile such as in the following example code
+::
+   
+     FC = mpif90
+     BUILD = Release
+
+     DECOMP_ROOT = /path/to/2decomp-fft
+     DECOMP_BUILD_DIR = $(DECOMP_ROOT)/build
+     DECOMP_INSTALL_DIR ?= $(DECOMP_BUILD_DIR)/opt # Use default unless set by user
+     
+     INC += -I$(DECOMP_INSTALL_DIR)/include
+     
+     # Users build/link targets
+     LIBS = -L$(DECOMP_INSTALL_DIR)/lib64 -L$(DECOMP_INSTALL_DIR)/lib -ldecomp2d
+     
+     # Building libdecomp.a
+     $(DECOMP_INSTALL_DIR)/lib/libdecomp.a:
+     FC=$(FC) cmake -S $(DECOMP_ROOT) -B $(DECOMP_BUILD_DIR) -DCMAKE_BUILD_TYPE=$(BUILD) -DCMAKE_INSTALL_PREFIX=$(DECOMP_INSTALL_DIR)
+     cmake --build $(DECOMP_BUILD_DIR) --target decomp2d
+     cmake --build $(DECOMP_BUILD_DIR) --target install
+     
+     # Clean libdecomp.a
+     clean-decomp:
+     cmake --build $(DECOMP_BUILD_DIR) --target clean
+     rm -f $(DECOMP_INSTALL_DIR)/lib/libdecomp.a
+     
 Profiling
 =========
 
 Profiling can be activated via `cmake` configuration, the recommended approach is to run the initial configuration as follows:
-```
-export caliper_DIR=/path/to/caliper/install/share/cmake/caliper
+
+``export caliper_DIR=/path/to/caliper/install/share/cmake/caliper
 export CXX=mpicxx
-cmake -S $path_to_sources -B $path_to_build_directory -DENABLE_PROFILER=caliper
-```
+cmake -S $path_to_sources -B $path_to_build_directory -DENABLE_PROFILER=caliper``
+
 where `ENABLE_PROFILER` is set to the profiling tool desired, currently supported values are: `caliper`.
 Note that when using `caliper` a C++ compiler is required as indicated in the above command line.
 
@@ -167,45 +177,45 @@ Miscellaneous
 List of preprocessor variables
 ==============================
 
-#### DEBUG
+**DEBUG**
 
 This variable is automatically added in debug and dev builds. Extra information is printed when it is present.
 
-#### DOUBLE_PREC
+**DOUBLE_PREC**
 
 When this variable is not present, the library uses single precision. When it is present, the library uses double precision. This preprocessor variable is driven by the CMake on/off variable `DOUBLE_PRECISION`.
 
-#### SAVE_SINGLE
+**SAVE_SINGLE**
 
 This variable is valid for double precision builds only. When it is present, snapshots are written in single precision. This preprocessor variable is driven by the CMake on/off variable `SINGLE_PRECISION_OUTPUT`.
 
-#### PROFILER
+**PROFILER**
 
 This variable is automatically added when selecting the profiler. It activates the profiling sections of the code.
 
-#### EVEN
+**EVEN**
 
 This preprocessor variable is not valid for GPU builds. It leads to padded alltoall operations. This preprocessor variable is driven by the CMake on/off variable `EVEN`.
 
-#### OVERWRITE
+**OVERWRITE**
 
 This variable leads to overwrite the input array when computing FFT. The support of this flag does not always correspond to in-place transforms, depending on the FFT backend selected, as described above. This preprocessor variable is driven by the CMake on/off variable `ENABLE_INPLACE`.
 
-#### HALO_DEBUG
+**HALO_DEBUG**
 
 This variable is used to debug the halo operations. This preprocessor variable is driven by the CMake on/off variable `HALO_DEBUG`.
 
-#### _GPU
+**_GPU**
 
 This variable is automatically added in GPU builds.
 
-#### _NCCL
+**_NCCL**
 
 This variable is valid only for GPU builds. The NVIDIA Collective Communication Library (NCCL) implements multi-GPU and multi-node communication primitives optimized for NVIDIA GPUs and Networking.
 
-## Optional dependencies
+**Optional dependencies**
 
-### FFTW
+**FFTW**
 
 The library [fftw](http://www.fftw.org/index.html) can be used as a backend for the FFT engine. The version 3.3.10 was tested, is supported and can be downloaded [here](http://www.fftw.org/download.html). Please note that one should build fftw and decomp2d against the same compilers. For build instructions, please check [here](http://www.fftw.org/fftw3_doc/Installation-on-Unix.html). Below is a suggestion for the compilation of the library in double precision (add `--enable-single` for a single precision build):
 
@@ -219,14 +229,13 @@ make -j check
 make install
 ```
 Please note that the resulting build is not compatible with CMake (https://github.com/FFTW/fftw3/issues/130). As a workaround, one can open the file `/path/to/fftw3/install/lib/cmake/fftw3/FFTW3Config.cmake` and comment the line
-```
-include ("${CMAKE_CURRENT_LIST_DIR}/FFTW3LibraryDepends.cmake")
-```
+
+``include ("${CMAKE_CURRENT_LIST_DIR}/FFTW3LibraryDepends.cmake")``
 
 To build `2decomp&fft` against fftw3, one can provide the package configuration for fftw3 in the `PKG_CONFIG_PATH` environment variable, this should be found under `/path/to/fftw3/install/lib/pkgconfig`. One can also provide the option `-DFFTW_ROOT=/path/to/fftw3/install`. Then either specify on the command line when configuring the build
-```
-cmake -S . -B build -DFFT_Choice=<fftw|fftw_f03> -DFFTW_ROOT=/path/to/fftw3/install
-```
+
+``cmake -S . -B build -DFFT_Choice=<fftw|fftw_f03> -DFFTW_ROOT=/path/to/fftw3/install``
+
 or modify the build configuration using `ccmake`.
 
 Note the legacy `fftw` interface lacks interface definitions and will fail when stricter compilation flags are used (e.g. when `-DCMAKE_BUILD_TYPE=Dev`) for this it is recommended to use `fftw_f03` which provides proper interfaces.
@@ -236,21 +245,20 @@ Caliper
 
 The library [caliper](https://github.com/LLNL/Caliper) can be used to profile the execution of the code. The version 2.9.1 was tested and is supported, version 2.8.0 has also been tested and is still expected to work. Please note that one must build caliper and decomp2d against the same C/C++/Fortran compilers and MPI libray. For build instructions, please check [here](https://github.com/LLNL/Caliper#building-and-installing) and [here](https://software.llnl.gov/Caliper/CaliperBasics.html#build-and-install). Below is a suggestion for the compilation of the library using the GNU compilers:
 
-```
-git clone https://github.com/LLNL/Caliper.git caliper_github
+``git clone https://github.com/LLNL/Caliper.git caliper_github
 cd caliper_github
 git checkout v2.9.1
 mkdir build && cd build
 cmake -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_Fortran_COMPILER=gfortran -DCMAKE_INSTALL_PREFIX=../../caliper_build_2.9.1 -DWITH_FORTRAN=yes -DWITH_MPI=yes -DBUILD_TESTING=yes ../
 make -j
 make test
-make install
-```
+make install``
 
 After installing Caliper ensure to set `caliper_DIR=/path/to/caliper/install/share/cmake/caliper`.
 Following this the `2decomp-fft` build can be configured to use Caliper profiling as
-```
-cmake -S . -B -DENABLE_PROFILER=caliper
-```
+
+``cmake -S . -B -DENABLE_PROFILER=caliper``
+
 or by modifying the configuration to set `ENABLE_PROFILER=caliper` via `ccmake`.
 
+
