Add two LCAO base group GPU version compilation options in toolchain #6014
Conversation
mohanchen
added
the
Compile & CICD & Docs & Dependencies
|
I'll submit a modification. After that, the ELPA-GPU compilation can be smoothly done in any cuda>11.6 environment. the installation and usage of cusolvermp needs more development |
- ELPA compiler flags modification - GPU_VER setting modification: user should specify the GPU compability number, but not the GPU name - Modify toolchain_[gnu,intel].sh and build_abacus_[gnu,intel].sh to use the above modification
LCAO-GPU modification
|
We will try to add cusolvermp installation & compilation inside toolchain |
Update: The method in deploying cusovermp and the related dependencies （UCC/UCX/libcal）can be multiple (nvhpc-sdk/individual-package) , and different version of these package may have different path, making it very difficult to link them automatcally. Besides, if one wants to use the multiple-GPU calculation, simplily install HPC_SDK automatically is not enough, and the UCC/UCX/cusolvermp need to be complied/deployed according to the server setting. So, we will not add automatically download/link in toolchain now. However, we will add a simple deployment method via HPC-SDK in README for user as reference to install and use the cusolvermp themselves. |
update README and cusolvermp
QuantumMisaka
left a comment
QuantumMisaka
left a comment
There was a problem hiding this comment.
All tests passed. cusolvermp is too complicated to be incorporate
|
@mohanchen @dzzz2001 @goodchong We think this version of toolchain is enough now for a simple GPU-LCAO installation with CUDA and ELPA |
|
LGTM |
…eepmodeling#6014) * Add optional LCAO base GPU versions supported by cusolvermp * Add optional LCAO base GPU versions supported by elpa * Add optional LCAO base GPU versions supported by elpa * Add L40S as GPUVER value for sm_89 architecture * Delete a few lines of content to enable Nvidia to compile * Add a specified Fortran mpi compiler for elpa to use * Add CUDA path for use by ELPA-GPU * Add optional LCAO base GPU versions supported by elpa * Modify a small issue * Change to manually specifying the link libraries for CAL and cusolverMp * Add the use of 'cusolvermp' or 'elpa' methods to compile ABACUS GPU-LCAO * Add the use of 'cusolvermp' or 'elpa' methods to compile ABACUS GPU-LCAO * Add the use of 'cusolvermp' or 'elpa' methods to compile ABACUS GPU-LCAO * Add modification - ELPA compiler flags modification - GPU_VER setting modification: user should specify the GPU compability number, but not the GPU name - Modify toolchain_[gnu,intel].sh and build_abacus_[gnu,intel].sh to use the above modification * minor adjustment * update README * give back cmake default option * update README and cusolvermp * Update README.md --------- Co-authored-by: JamesMisaka <[email protected]>
…6014) * Add optional LCAO base GPU versions supported by cusolvermp * Add optional LCAO base GPU versions supported by elpa * Add optional LCAO base GPU versions supported by elpa * Add L40S as GPUVER value for sm_89 architecture * Delete a few lines of content to enable Nvidia to compile * Add a specified Fortran mpi compiler for elpa to use * Add CUDA path for use by ELPA-GPU * Add optional LCAO base GPU versions supported by elpa * Modify a small issue * Change to manually specifying the link libraries for CAL and cusolverMp * Add the use of 'cusolvermp' or 'elpa' methods to compile ABACUS GPU-LCAO * Add the use of 'cusolvermp' or 'elpa' methods to compile ABACUS GPU-LCAO * Add the use of 'cusolvermp' or 'elpa' methods to compile ABACUS GPU-LCAO * Add modification - ELPA compiler flags modification - GPU_VER setting modification: user should specify the GPU compability number, but not the GPU name - Modify toolchain_[gnu,intel].sh and build_abacus_[gnu,intel].sh to use the above modification * minor adjustment * update README * give back cmake default option * update README and cusolvermp * Update README.md --------- Co-authored-by: JamesMisaka <[email protected]>
3 participants
Firstly, it's cusolverMp merhod
Change the link libraries for the cal and cusolverMp sections in the CMakeLists.txt file to manually specified, and then manually pass the parameters in
cmake -D CAL_CUSOLVERMP_PATH=/path/to/libAdd the following options after build_abacus_gnu.sh
-DUSE_CUDA=ON \-DENABLE_CUSOLVERMP=ON \-D CAL_CUSOLVERMP_PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/24.11/math_libs/12.6/targets/x86_64-linux/libCAL_CUSOLVERMP-PATH needs to be set according to different environments
next it is necessary to set up the environment for cal and hpcx, which can be added to~/. bashrc or manually set up an env.sh
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/hpc_sdk/Linux_x86_64/24.11/comm_libs/12.6/hpcx/hpcx-2.20/ucc/libexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/hpc_sdk/Linux_x86_64/24.11/comm_libs/12.6/hpcx/hpcx-2.20/ucx/libexport CPATH=$CPATH:/opt/nvidia/hpc_sdk/Linux_x86_64/24.11/math_libs/12.6/targets/x86_64-linux/includeThe second is the ELPA method
Add the following options after toolchain_gnu.sh
export CUDA_PATH=/usr/local/cuda--enable-cuda \--gpu-ver=89 \The 40 Series here is newly added in the install_abacus_toolchain.sh file, corresponding to sm_89
The above two methods can be compiled successfully by using the build_abacus_gnu.sh file