Toolchain update: use new intel-OneAPI compiler as default (#5458)

* ifx add and minor fix

* set default --with-intel-classic=no

* ifx link add-in

* support new intelmpi compiler

* date info

* minor update

* Fix --with-intelmpi-classic=yes problem

* fix mpiicpx&mpiicx link error

* Truly fixed

* minor adjustment

Author: QuantumMisaka

toolchain/README.md

@@ -31,7 +31,6 @@ and give setup files that you can use to compile ABACUS.
 - [ ] A better README and Detail markdown file.
 - [ ] Automatic installation of [DEEPMD](https://github.com/deepmodeling/deepmd-kit).
 - [ ] Better compliation method for ABACUS-DEEPMD and ABACUS-DEEPKS.
-- [ ] A better `setup` and toolchain code structure.
 - [ ] Modulefile generation scripts.
 - [ ] Support for AMD compiler and math lib like `AOCL` and `AOCC`
 
@@ -54,19 +53,23 @@ There are also well-modified script to run *install_abacus_toolchain.sh* for `gn
 > ./toolchain_intel-mpich.sh
 ```
 
-It is recommended to run them first to get a fast installation of ABACUS under certain environments.
+It is recommended to run one of them first to get a fast installation of ABACUS under certain environments.
 
-If you have a fresh environments and you have `sudo` permission, you can use *install_requirements.sh* to install system libraries and dependencies needed by toolchain.
+If you are using Intel environments via Intel-OneAPI: please note:
+1. After version 2024.0, Intel classic compilers `icc` and `icpc` are not present, so as `ifort` after version 2025.0. Intel MPI compiler will also be updated to `mpiicx`, `mpiicpx` and `mpiifx`.
+2. toolchain will detect `icx`, `icpx`, `ifx`, `mpiicx`, `mpiicpx` and `mpiifx` as default compiler.
+3. Users can manually specify `--with-intel-classic=yes` to use Intel classic compiler in `toolchain*.sh`, or specify `--with-intel-mpi-clas=yes` to use Intel MPI classic compiler in `toolchain*.sh` while keep the CC, CXX and F90 compiler to new version.
+4. Users can manually specify `--with-ifx=no` in `toolchain*.sh` to use `ifort` while keep other compiler to new version. 
+5. More information is in the later part of this README.
+
+**Notice: You GCC version should be no lower than 5 !!!, larger than 7.3.0 is recommended**
 
 **Notice: You SHOULD `source` or `module load` related environments before use toolchain method for installation, espacially for `gcc` or `intel-oneAPI` !!!! for example, `module load mkl mpi icc compiler`**
 
 **Notice: You SHOULD keep your environments systematic, for example, you CANNOT load `intel-OneAPI` environments while use gcc toolchain !!!**
 
 **Notice: If your server system already have libraries like `cmake`, `openmpi`, please change related setting in `toolchain*.sh` like `--with-cmake=system`**
 
-```shell
-> ./install_ABACUS_toolchain.sh
-```
 
 All packages will be downloaded from [cp2k-static/download](https://www.cp2k.org/static/downloads). by  `wget` , and will be detailedly compiled and installed in `install` directory by toolchain scripts, despite of:
 
@@ -198,6 +201,7 @@ cmake -B $BUILD_DIR -DCMAKE_INSTALL_PREFIX=$PREFIX \
 
 Notice: You CANNOT use `icpx` compiler for GPU version of ABACUS for now, see discussion here [#2906](https://github.com/deepmodeling/abacus-develop/issues/2906) and [#4976](https://github.com/deepmodeling/abacus-develop/issues/4976)
 
+If you wants to use ABACUS GPU-LCAO by `cusolvermp` or `elpa`, please contact the coresponding developer, toolchain do not fully support them now.
 
 ### Shell problem
 
@@ -238,15 +242,17 @@ When you encounter problem like `GLIBCXX_3.4.29 not found`, it is sure that your
 
 After my test, you need `gcc`>11.3.1 to enable deepmd feature in ABACUS.
 
-### ELPA problem via Intel-oneAPI toolchain in AMD server
+### Intel-oneAPI problem
+
+#### ELPA problem via Intel-oneAPI toolchain in AMD server
 
 The default compiler for Intel-oneAPI is `icpx` and `icx`, which will cause problem when compling ELPA in AMD server. (Which is a problem and needed to have more check-out)
 
 The best way is to change `icpx` to `icpc`, `icx` to `icc`. user can manually change it in toolchain*.sh via `--with-intel-classic=yes`
 
 Notice: `icc` and `icpc` from Intel Classic Compiler of Intel-oneAPI is not supported for 2024.0 and newer version. And Intel-OneAPI 2023.2.0 can be found in website. See discussion here [#4976](https://github.com/deepmodeling/abacus-develop/issues/4976)
 
-### Intel-oneAPI problem
+#### link problem in early 2023 version oneAPI
 
 Sometimes Intel-oneAPI have problem to link `mpirun`, 
 which will always show in 2023.2.0 version of MPI in Intel-oneAPI. 

toolchain/build_abacus_gnu.sh

@@ -65,7 +65,7 @@ cmake -B $BUILD_DIR -DCMAKE_INSTALL_PREFIX=$PREFIX \
 # for -lmkl when load libtorch
 # module load mkl
 
-# if one want's to include deepmd, your gcc version should be >= 11.3.0
+# if one want's to include deepmd, your system gcc version should be >= 11.3.0 for glibc requirements
 
 cmake --build $BUILD_DIR -j `nproc` 
 cmake --install $BUILD_DIR 2>/dev/null

toolchain/build_abacus_intel-mpich.sh

@@ -56,7 +56,7 @@ cmake -B $BUILD_DIR -DCMAKE_INSTALL_PREFIX=$PREFIX \
 #       -DTensorFlow_DIR=$DEEPMD \
 
 
-# if one want's to include deepmd, your gcc version should be >= 11.3.0
+# if one want's to include deepmd, your system gcc version should be >= 11.3.0 for glibc requirements
 
 cmake --build $BUILD_DIR -j `nproc` 
 cmake --install $BUILD_DIR 2>/dev/null

toolchain/build_abacus_intel.sh

@@ -36,7 +36,7 @@ RAPIDJSON=$INSTALL_DIR/rapidjson-1.1.0/
 # if use deepks and deepmd
 cmake -B $BUILD_DIR -DCMAKE_INSTALL_PREFIX=$PREFIX \
         -DCMAKE_CXX_COMPILER=icpx \
-        -DMPI_CXX_COMPILER=mpiicpc \
+        -DMPI_CXX_COMPILER=mpiicpx \
         -DMKLROOT=$MKLROOT \
         -DELPA_DIR=$ELPA \
         -DCEREAL_INCLUDE_DIR=$CEREAL \
@@ -60,7 +60,7 @@ cmake -B $BUILD_DIR -DCMAKE_INSTALL_PREFIX=$PREFIX \
 cmake --build $BUILD_DIR -j `nproc` 
 cmake --install $BUILD_DIR 2>/dev/null
 
-# if one want's to include deepmd, your gcc version should be >= 11.3.0
+# if one want's to include deepmd, your system gcc version should be >= 11.3.0 for glibc requirements
 
 # generate abacus_env.sh
 cat << EOF > "${TOOL}/abacus_env.sh"

toolchain/install_abacus_toolchain.sh

@@ -97,11 +97,11 @@ OPTIONS:
                           options to values other than no will also switch --mpi-mode
                           to the respective mode.
 --math-mode               Selects which core math library to use. Available options
-                          are: acml, cray, mkl, and openblas. The option "cray"
+                          are: cray, mkl, and openblas. The option "cray"
                           corresponds to cray libsci, and is the default for CRAY
                           (CLE) systems. For non-CRAY systems, if env variable MKLROOT
                           exists then mkl will be default, otherwise openblas is the
-                          default option. Explicitly setting --with-acml, --with-mkl,
+                          default option. Explicitly setting --with-mkl,
                           or --with-openblas options will switch --math-mode to the
                           respective modes.
 --gpu-ver                 Selects the GPU architecture for which to compile. Available
@@ -152,8 +152,10 @@ The --with-PKG options follow the rules:
                           Default = system
   --with-intel            Use the Intel compiler to compile ABACUS.
                           Default = system
-  --with-intel-classic    Use the classic Intel compiler to compile ABACUS.
+  --with-intel-classic    Use the classic Intel compiler (icc, icpc, ifort) to compile ABACUS.
                           Default = no
+  --with-ifx              Use the new Intel Fortran compiler ifx instead of ifort to compile dependence of ABACUS, along with mpiifx (if --with-intel-classic=no)
+                          Default = yes
   --with-cmake            Cmake utilities
                           Default = install
   --with-openmpi          OpenMPI, important if you want a parallel version of ABACUS.
@@ -166,6 +168,8 @@ The --with-PKG options follow the rules:
   --with-intelmpi         Intel MPI, MPI library like OpenMPI. one should
                           use only one of OpenMPI, MPICH or Intel MPI.
                           Default = system
+  --with-intel-mpi-clas   Use the classic Intelmpi compiler (mpiicc, mpiicpc and mpiifort)
+                          Default = no
   --with-libxc            libxc, exchange-correlation library. Needed for
                           QuickStep DFT and hybrid calculations.
                           Default = install
@@ -231,7 +235,7 @@ EOF
 # ------------------------------------------------------------------------
 tool_list="gcc intel cmake"
 mpi_list="mpich openmpi intelmpi"
-math_list="mkl acml openblas"
+math_list="mkl openblas"
 lib_list="fftw libxc scalapack elpa cereal rapidjson libtorch libnpy libri libcomm"
 package_list="${tool_list} ${mpi_list} ${math_list} ${lib_list}"
 # ------------------------------------------------------------------------
@@ -262,7 +266,6 @@ if [ "${MKLROOT}" ]; then
 else
   export MATH_MODE="openblas"
 fi
-with_acml="__SYSTEM__"
 with_openblas="__INSTALL__"
 with_elpa="__INSTALL__"
 with_cereal="__INSTALL__"
@@ -305,13 +308,15 @@ enable_opencl="__FALSE__"
 enable_cuda="__FALSE__"
 enable_hip="__FALSE__"
 export intel_classic="no" 
-# no, then icc->icx, icpc->icpx, 
+# no, then icc->icx, icpc->icpx
 # which cannot compile elpa in AMD server
 # due to some so-called cross-compile problem
 # and will lead to problem in force calculation
 # but icx is recommended by intel compiler
 # option: --with-intel-classic can change it to yes/no
-# zhaoqing by 2023.08.31
+# zhaoqing by 2023.08
+export intelmpi_classic="no"
+export with_ifx="yes"
 export GPUVER="no"
 export MPICH_DEVICE="ch4"
 export TARGET_CPU="native"
@@ -406,15 +411,12 @@ while [ $# -ge 1 ]; do
         mkl)
           export MATH_MODE="mkl"
           ;;
-        acml)
-          export MATH_MODE="acml"
-          ;;
         openblas)
           export MATH_MODE="openblas"
           ;;
         *)
           report_error ${LINENO} \
-            "--math-mode currently only supports mkl, acml, and openblas as options"
+            "--math-mode currently only supports mkl, and openblas as options"
           ;;
       esac
       ;;
@@ -507,11 +509,17 @@ while [ $# -ge 1 ]; do
       fi
       ;;
     --with-intel-classic*)
-      intel_classic=$(read_with "${1}" "yes") # default yes
+      intel_classic=$(read_with "${1}" "no") # default new intel compiler
+      ;;
+    --with-intel-mpi-clas*)
+      intelmpi_classic=$(read_with "${1}" "no") # default new intel mpi compiler
       ;;
     --with-intel*)
       with_intel=$(read_with "${1}" "__SYSTEM__")
       ;;
+    --with-ifx*)
+      with_ifx=$(read_with "${1}" "yes") # default yes
+      ;;
     --with-libxc*)
       with_libxc=$(read_with "${1}")
       ;;
@@ -524,12 +532,6 @@ while [ $# -ge 1 ]; do
         export MATH_MODE="mkl"
       fi
       ;;
-    --with-acml*)
-      with_acml=$(read_with "${1}")
-      if [ "${with_acml}" != "__DONTUSE__" ]; then
-        export MATH_MODE="acml"
-      fi
-      ;;
     --with-openblas*)
       with_openblas=$(read_with "${1}")
       if [ "${with_openblas}" != "__DONTUSE__" ]; then

toolchain/scripts/stage0/install_gcc.sh

@@ -11,8 +11,8 @@ SCRIPT_DIR="$(cd "$(dirname "$SCRIPT_NAME")/.." && pwd -P)"
 gcc_ver="13.2.0"
 gcc_sha256="8cb4be3796651976f94b9356fa08d833524f62420d6292c5033a9a26af315078"
 # use gcc 14 with caution
-#gcc_ver="14.1.0"
-#gcc_sha256="a0be066c02775002a0fa65ad3c65fb56a8bfd923d072a26ed148c0439ecdb68f"
+#gcc_ver="14.2.0"
+#gcc_sha256="7d376d445f93126dc545e2c0086d0f647c3094aae081cdb78f42ce2bc25e7293"
 
 source "${SCRIPT_DIR}"/common_vars.sh
 source "${SCRIPT_DIR}"/tool_kit.sh

toolchain/scripts/stage0/install_intel.sh

@@ -3,7 +3,7 @@
 # TODO: Review and if possible fix shellcheck errors.
 # shellcheck disable=all
 
-# Last Update in 2024-0811
+# Last Update in 2024-1112
 
 [ "${BASH_SOURCE[0]}" ] && SCRIPT_NAME="${BASH_SOURCE[0]}" || SCRIPT_NAME=${0}
 SCRIPT_DIR="$(cd "$(dirname "${SCRIPT_NAME}")/.." && pwd -P)"
@@ -37,7 +37,11 @@ case "${with_intel}" in
     else
       check_command icx "intel" && CC="$(realpath $(command -v icx))" || exit 1
       check_command icpx "intel" && CXX="$(realpath $(command -v icpx))" || exit 1
-      check_command ifort "intel" && FC="$(realpath $(command -v ifort))" || exit 1
+      if [ "${with_ifx}" = "yes" ]; then
+        check_command ifx "intel" && FC="$(realpath $(command -v ifx))" || exit 1
+      else
+        check_command ifort "intel" && FC="$(realpath $(command -v ifort))" || exit 1
+      fi
     fi
     F90="${FC}"
     F77="${FC}"
@@ -58,8 +62,11 @@ case "${with_intel}" in
     else
       check_command ${pkg_install_dir}/bin/icx "intel" && CC="${pkg_install_dir}/bin/icx" || exit 1
       check_command ${pkg_install_dir}/bin/icpx "intel" && CXX="${pkg_install_dir}/bin/icpx" || exit 1
-      check_command ${pkg_install_dir}/bin/ifort "intel" && FC="${pkg_install_dir}/bin/ifort" || exit 1
-      # is ifx needed ?
+      if [ "${with_ifx}" = "yes" ]; then
+        check_command ${pkg_install_dir}/bin/ifx "intel" && FC="${pkg_install_dir}/bin/ifx" || exit 1
+      else
+        check_command ${pkg_install_dir}/bin/ifort "intel" && FC="${pkg_install_dir}/bin/ifort" || exit 1
+      fi
     fi
     F90="${FC}"
     F77="${FC}"

toolchain/scripts/stage1/install_intelmpi.sh

@@ -3,7 +3,7 @@
 # TODO: Review and if possible fix shellcheck errors.
 # shellcheck disable=all
 
-# Last Update in 2023-0918
+# Last Update in 2024-1112
 
 [ "${BASH_SOURCE[0]}" ] && SCRIPT_NAME="${BASH_SOURCE[0]}" || SCRIPT_NAME=$0
 SCRIPT_DIR="$(cd "$(dirname "$SCRIPT_NAME")/.." && pwd -P)"
@@ -32,10 +32,24 @@ case "${with_intelmpi}" in
   __SYSTEM__)
     echo "==================== Finding Intel MPI from system paths ===================="
     check_command mpiexec "intelmpi" && MPIRUN="$(realpath $(command -v mpiexec))"
+    if [ "${intel_classic}" = "yes" ]; then
+    # if intel compiler used as classic, so as intelmpi
+        export intelmpi_classic="yes"
+    fi
     if [ "${with_intel}" != "__DONTUSE__" ]; then
-      check_command mpiicc "intelmpi" && MPICC="$(realpath $(command -v mpiicc))" || exit 1
-      check_command mpiicpc "intelmpi" && MPICXX="$(realpath $(command -v mpiicpc))" || exit 1
-      check_command mpiifort "intelmpi" && MPIFC="$(realpath $(command -v mpiifort))" || exit 1
+        if [ "${intelmpi_classic}" = "yes" ]; then
+            check_command mpiicc "intelmpi" && MPICC="$(realpath $(command -v mpiicc))" || exit 1
+            check_command mpiicpc "intelmpi" && MPICXX="$(realpath $(command -v mpiicpc))" || exit 1
+            check_command mpiifort "intelmpi" && MPIFC="$(realpath $(command -v mpiifort))" || exit 1
+        else
+            check_command mpiicx "intelmpi" && MPICC="$(realpath $(command -v mpiicx))" || exit 1
+            check_command mpiicpx "intelmpi" && MPICXX="$(realpath $(command -v mpiicpx))" || exit 1
+            if [ "${with_ifx}" == "yes" ]; then
+                check_command mpiifx "intelmpi" && MPIFC="$(realpath $(command -v mpiifx))" || exit 1
+            else
+                check_command mpiifort "intelmpi" && MPIFC="$(realpath $(command -v mpiifort))" || exit 1
+            fi
+        fi
     else
       echo "The use of Intel MPI is only supported with the Intel compiler"
       exit 1
@@ -59,17 +73,27 @@ case "${with_intelmpi}" in
     check_dir "${pkg_install_dir}/include"
     check_command ${pkg_install_dir}/bin/mpiexec "intel" && MPIRUN="${pkg_install_dir}/bin/mpiexec" || exit 1
     if [ "${with_intel}" != "__DONTUSE__" ]; then
-      check_command ${pkg_install_dir}/bin/mpiicc "intel" && MPICC="${pkg_install_dir}/bin/mpiicc" || exit 1
-      check_command ${pkg_install_dir}/bin/mpiicpc "intel" && MPICXX="${pkg_install_dir}/bin/mpiicpc" || exit 1
-      check_command ${pkg_install_dir}/bin/mpiifort "intel" && MPIFC="${pkg_install_dir}/bin/mpiifort" || exit 1
+        if [ "${intelmpi_classic}" = "yes" ]; then
+            check_command ${pkg_install_dir}/bin/mpiicc "intel" && MPICC="${pkg_install_dir}/bin/mpiicc" || exit 1
+            check_command ${pkg_install_dir}/bin/mpiicpc "intel" && MPICXX="${pkg_install_dir}/bin/mpiicpc" || exit 1
+            check_command ${pkg_install_dir}/bin/mpiifort "intel" && MPIFC="${pkg_install_dir}/bin/mpiifort" || exit 1
+        else
+            check_command ${pkg_install_dir}/bin/mpiicx "intel" && MPICC="${pkg_install_dir}/bin/mpiicx" || exit 1
+            check_command ${pkg_install_dir}/bin/mpiicpx "intel" && MPICXX="${pkg_install_dir}/bin/mpiicpx" || exit 1
+            if [ "${with_ifx}" = "yes" ]; then
+                check_command ${pkg_install_dir}/bin/mpiifx "intel" && MPIFC="${pkg_install_dir}/bin/mpiifx" || exit 1
+            else
+                check_command ${pkg_install_dir}/bin/mpiifort "intel" && MPIFC="${pkg_install_dir}/bin/mpiifort" || exit 1
+            fi
+        fi
     else
       echo "The use of Intel MPI is only supported with the Intel compiler"
       exit 1
     fi
     MPIFORT="${MPIFC}"
     MPIF77="${MPIFC}"
     # include path is already handled by compiler wrapper scripts (can cause wrong mpi.mod with GNU Fortran)
-    #INTELMPI_CFLAGS="-I'${pkg_install_dir}/include'"
+    INTELMPI_CFLAGS="-I'${pkg_install_dir}/include'"
     INTELMPI_LDFLAGS="-L'${pkg_install_dir}/lib' -Wl,-rpath,'${pkg_install_dir}/lib'"
     ;;
 esac
@@ -81,7 +105,11 @@ if [ "${with_intelmpi}" != "__DONTUSE__" ]; then
   else
     I_MPI_CXX="icpx"
     I_MPI_CC="icx"
-    I_MPI_FC="ifort"
+    if [ "${with_ifx}" = "yes" ]; then
+      I_MPI_FC="ifx"
+    else
+      I_MPI_FC="ifort"
+    fi
   fi
   INTELMPI_LIBS="-lmpi -lmpicxx"
   echo "I_MPI_CXX is ${I_MPI_CXX}"

toolchain/scripts/stage4/install_libtorch.sh

@@ -44,7 +44,7 @@ case "${with_libtorch}" in
     filename="${dirname}.zip"
 
     if verify_checksums "${install_lock_file}"; then
-      echo "${filename} is already installed, skipping it."
+      echo "${dirname} is already installed, skipping it."
     else
         if [ -f ${filename} ]; then
             echo "${filename} is found"