Skip to content

Commit c64bc25

Browse files
dyzhengdyzheng
authored
Merge pull request #145 from dyzheng/v2.2.3
V2.2.3
2 parents 21ac83f + 6364f7a commit c64bc25

File tree

353 files changed

+25507
-6354
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

353 files changed

+25507
-6354
lines changed

‎.github/workflows/cuda.yml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
name: CUDA Test
2+
3+
on:
4+
workflow_dispatch:
5+
6+
jobs:
7+
start-runner:
8+
name: Start self-hosted EC2 runner
9+
runs-on: ubuntu-latest
10+
outputs:
11+
label: ${{ steps.start-ec2-runner.outputs.label }}
12+
ec2-instance-id: ${{ steps.start-ec2-runner.outputs.ec2-instance-id }}
13+
steps:
14+
- name: Configure AWS credentials
15+
uses: aws-actions/configure-aws-credentials@v1
16+
with:
17+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
18+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
19+
aws-region: us-east-2
20+
- name: Start EC2 runner
21+
id: start-ec2-runner
22+
uses: machulav/ec2-github-runner@v2
23+
with:
24+
mode: start
25+
github-token: ${{ secrets.PAT }}
26+
ec2-image-id: ami-04cd9fec4a7a39019
27+
ec2-instance-type: g4dn.xlarge
28+
subnet-id: subnet-72d3e53e
29+
security-group-id: sg-06b0c93122c08aeab
30+
31+
test:
32+
name: Do the job on the runner
33+
needs: start-runner # required to start the main job when the runner is ready
34+
runs-on: ${{ needs.start-runner.outputs.label }} # run the job on the newly created runner
35+
container:
36+
image: ghcr.io/deepmodeling/abacus-development-kit:cuda
37+
options: --gpus all
38+
steps:
39+
- name: Checkout
40+
uses: actions/checkout@v3
41+
- name: Build cuSolver
42+
run: |
43+
nvidia-smi
44+
cmake -B build -DUSE_CUSOLVER_LCAO=ON
45+
cmake --build build -j4
46+
cmake --install build
47+
- name: Test
48+
run: |
49+
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda/lib64
50+
cd tests/integrate
51+
echo "ks_solver cusolver" >> ./270_NO_MD_2O/INPUT
52+
./Autotest.sh -r 270_NO_MD_2O
53+
54+
stop-runner:
55+
name: Stop self-hosted EC2 runner
56+
needs:
57+
- start-runner # required to get output from the start-runner job
58+
- test # required to wait when the main job is done
59+
runs-on: ubuntu-latest
60+
if: ${{ always() }} # required to stop the runner even if the error happened in the previous jobs
61+
steps:
62+
- name: Configure AWS credentials
63+
uses: aws-actions/configure-aws-credentials@v1
64+
with:
65+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
66+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
67+
aws-region: us-east-2
68+
- name: Stop EC2 runner
69+
uses: machulav/ec2-github-runner@v2
70+
with:
71+
mode: stop
72+
github-token: ${{ secrets.PAT }}
73+
label: ${{ needs.start-runner.outputs.label }}
74+
ec2-instance-id: ${{ needs.start-runner.outputs.ec2-instance-id }}

‎.github/workflows/container.yml‎ renamed to ‎.github/workflows/devcontainer.yml‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ on:
88
jobs:
99
build_container_and_push:
1010
runs-on: ubuntu-latest
11+
if: github.repository_owner == 'deepmodeling'
1112
strategy:
1213
matrix:
1314
dockerfile: ["gnu","intel","cuda"]
@@ -25,10 +26,19 @@ jobs:
2526
username: ${{ github.actor }}
2627
password: ${{ secrets.GITHUB_TOKEN }}
2728

29+
- name: Login to Aliyun Registry
30+
uses: docker/login-action@v1
31+
with:
32+
registry: ${{ secrets.DP_HARBOR_REGISTRY }}
33+
username: ${{ secrets.DP_HARBOR_USERNAME }}
34+
password: ${{ secrets.DP_HARBOR_PASSWORD }}
35+
2836
- name: Build and Push Container
2937
uses: docker/build-push-action@v2
3038
with:
31-
tags: ghcr.io/${{ github.repository_owner }}/abacus-development-kit:${{ matrix.dockerfile }}
39+
tags: |
40+
ghcr.io/${{ github.repository_owner }}/abacus-development-kit:${{ matrix.dockerfile }}
41+
${{ secrets.DP_HARBOR_REGISTRY }}/dplc/abacus-${{ matrix.dockerfile }}:latest
3242
file: Dockerfile.${{ matrix.dockerfile }}
3343
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/abacus-development-kit:${{matrix.dockerfile}}
3444
cache-to: type=inline

‎.github/workflows/image.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
name: Build Image
2+
3+
on:
4+
push:
5+
branches:
6+
- develop
7+
8+
jobs:
9+
build_container_and_push:
10+
runs-on: ubuntu-latest
11+
if: github.repository_owner == 'deepmodeling'
12+
steps:
13+
- name: Checkout
14+
uses: actions/checkout@v3
15+
16+
- name: Setup Docker Buildx
17+
uses: docker/setup-buildx-action@v1
18+
19+
- name: Login to GitHub Container Registry
20+
uses: docker/login-action@v1
21+
with:
22+
registry: ghcr.io
23+
username: ${{ github.actor }}
24+
password: ${{ secrets.GITHUB_TOKEN }}
25+
26+
- name: Login to Aliyun Registry
27+
uses: docker/login-action@v1
28+
with:
29+
registry: ${{ secrets.DP_HARBOR_REGISTRY }}
30+
username: ${{ secrets.DP_HARBOR_USERNAME }}
31+
password: ${{ secrets.DP_HARBOR_PASSWORD }}
32+
33+
- name: Build and Push Container
34+
uses: docker/build-push-action@v2
35+
with:
36+
tags: |
37+
ghcr.io/${{ github.repository_owner }}/abacus:latest
38+
${{ secrets.DP_HARBOR_REGISTRY }}/dplc/abacus:latest
39+
file: Dockerfile
40+
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/abacus:latest
41+
cache-to: type=inline
42+
push: true

‎.github/workflows/performance.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Performance test
2+
3+
on:
4+
workflow_dispatch:
5+
6+
jobs:
7+
test:
8+
name: Performance test
9+
runs-on: self-hosted
10+
strategy:
11+
matrix:
12+
tag: ["gnu", "intel"]
13+
if: github.repository_owner == 'deepmodeling'
14+
container: ghcr.io/${{ github.repository_owner }}/abacus-development-kit:${{ matrix.tag }}
15+
timeout-minutes: 2880
16+
steps:
17+
- name: Checkout
18+
uses: actions/checkout@v3
19+
- name: Install Requirements
20+
run: |
21+
apt install -y time
22+
- name: Build
23+
run: |
24+
cmake -B build -DENABLE_LIBXC=ON
25+
cmake --build build -j16
26+
cmake --install build
27+
- name: Test
28+
run: |
29+
. /opt/intel/oneapi/setvars.sh 2> /dev/null || :
30+
cd examples/performance/
31+
bash run.sh
32+
- name: Show Result
33+
run: |
34+
cat examples/performance/sumall.dat

‎.github/workflows/test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ jobs:
1616
ref: "refs/pull/${{ github.event.number }}/merge"
1717
- name: Build
1818
run: |
19-
cmake -B build -DBUILD_TESTING=ON -DENABLE_DEEPKS=ON
19+
cmake -B build -DBUILD_TESTING=ON -DENABLE_DEEPKS=ON -DENABLE_LIBXC=ON
2020
cmake --build build -j8
2121
cmake --install build
2222
- name: Test

‎CMakeLists.txt‎

Lines changed: 37 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
########################################
22
# CMake build system
33
# This file is part of ABACUS
4-
cmake_minimum_required(VERSION 3.18)
4+
cmake_minimum_required(VERSION 3.16)
55
########################################
66

77
project(ABACUS
@@ -13,15 +13,16 @@ project(ABACUS
1313

1414
option(ENABLE_DEEPKS "Enable DeePKS functionality" OFF)
1515
option(ENABLE_LIBXC "Enable LibXC functionality" OFF)
16-
option(USE_CUDA "Enable support to CUDA." OFF)
16+
option(USE_CUDA "Enable support to CUDA for PW." OFF)
17+
option(USE_CUSOLVER_LCAO "Enable support to CUSOLVER for LCAO." OFF)
1718
option(USE_ROCM "Enable support to ROCm." OFF)
1819
option(USE_OPENMP " Enable OpenMP in abacus." ON)
1920
option(ENABLE_ASAN "Enable AddressSanitizer" OFF)
2021
option(BUILD_TESTING "Build ABACUS unit tests" OFF)
2122
option(GENERATE_TEST_REPORTS "Enable test report generation" OFF)
2223

2324
set(ABACUS_BIN_NAME abacus)
24-
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/modules)
25+
list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/modules)
2526
set(ABACUS_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/source)
2627
set(ABACUS_TEST_DIR ${CMAKE_CURRENT_SOURCE_DIR}/tests)
2728
set(ABACUS_BIN_PATH ${CMAKE_CURRENT_BINARY_DIR}/${ABACUS_BIN_NAME})
@@ -58,21 +59,33 @@ set(CMAKE_CXX_STANDARD 11)
5859
include(CheckLanguage)
5960
check_language(CUDA)
6061
if(CMAKE_CUDA_COMPILER)
61-
if(NOT DEFINED USE_CUDA)
62-
message("CUDA components detected. \nWill build the CUDA version of ABACUS.")
63-
set(USE_CUDA ON)
62+
if(NOT DEFINED USE_CUDA OR NOT DEFINED USE_CUSOLVER_LCAO)
63+
if (NOT DEFINED USE_CUDA AND NOT DEFINED USE_CUSOLVER_LCAO)
64+
message("CUDA components detected. \nWill build the CUDA for PW version of ABACUS by default.")
65+
set(USE_CUDA ON)
66+
set(USE_CUSOLVER_LCAO OFF)
67+
elseif (NOT DEFINED USE_CUDA)
68+
set(USE_CUDA OFF)
69+
else()
70+
set(USE_CUSOLVER_LCAO OFF)
71+
endif()
6472
else()
65-
if(NOT USE_CUDA)
66-
message(WARNING "CUDA components detected, but USE_CUDA set to OFF. \nNOT building CUDA version of ABACUS.")
73+
if(NOT USE_CUDA AND NOT USE_CUSOLVER_LCAO)
74+
message(WARNING "CUDA components detected, but both USE_CUDA and USE_CUSOLVER_LCAO set to OFF. \nNOT building CUDA version of ABACUS.")
75+
elseif (USE_CUDA AND USE_CUSOLVER_LCAO)
76+
message(FATAL_ERROR "USE_CUDA and USE_CUSOLVER_LCAO set, but now they not allowed to coexist.")
6777
endif()
6878
endif()
6979
else() # CUDA not found
70-
if (USE_CUDA)
71-
message(FATAL_ERROR "USE_CUDA set but no CUDA components found.")
80+
if (USE_CUDA OR USE_CUSOLVER_LCAO)
81+
message(FATAL_ERROR "USE_CUDA or USE_CUSOLVER_LCAO set but no CUDA components found.")
7282
set(USE_CUDA OFF)
83+
set(USE_CUSOLVER_LCAO OFF)
7384
endif()
7485
endif()
75-
if(USE_CUDA)
86+
87+
if(USE_CUDA OR USE_CUSOLVER_LCAO)
88+
cmake_minimum_required(VERSION 3.18) # required by `CUDA_ARCHITECTURES` below
7689
set(CMAKE_CXX_STANDARD 14)
7790
set(CMAKE_CXX_EXTENSIONS ON)
7891
set(CMAKE_CXX_STANDARD_REQUIRED ON)
@@ -92,11 +105,18 @@ if(USE_CUDA)
92105
60 # P100
93106
70 # V100
94107
75 # T4
108+
80 # A100
95109
)
96110
include_directories(${CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES})
97-
add_compile_definitions(__CUDA)
111+
if (USE_CUDA)
112+
add_compile_definitions(__CUDA)
113+
endif()
114+
if (USE_CUSOLVER_LCAO)
115+
add_compile_definitions(__CUSOLVER_LCAO)
116+
endif()
98117
endif()
99118

119+
100120
# Warning: CMake add support to HIP in version 3.21. This is rather a new version.
101121
# Use cmake with AMD-ROCm: https://rocmdocs.amd.com/en/latest/Installation_Guide/Using-CMake-with-AMD-ROCm.html
102122
if(USE_ROCM)
@@ -263,6 +283,7 @@ target_link_libraries(${ABACUS_BIN_NAME}
263283
cell
264284
symmetry
265285
md
286+
planewave
266287
surchem
267288
neighbor
268289
orb
@@ -276,6 +297,10 @@ target_link_libraries(${ABACUS_BIN_NAME}
276297
ri
277298
driver
278299
xc
300+
hsolver
301+
elecstate
302+
hamilt
303+
psi
279304
esolver
280305
-lm
281306
)

‎Dockerfile‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# To build this Dockerfile, run `docker build -t abacus - < Dockerfile`.
2+
# Pull image with `docker pull ghcr.io/deepmodeling/abacus:latest`.
3+
FROM ubuntu:latest
4+
RUN apt update && apt install -y --no-install-recommends libopenblas-dev liblapack-dev libscalapack-mpi-dev libelpa-dev libfftw3-dev libcereal-dev libxc-dev g++ make cmake bc time sudo vim git
5+
# If you wish to use the LLVM compiler, replace 'g++' above with 'clang libomp-dev'.
6+
RUN GIT_SSL_NO_VERIFY=true git clone https://github.com/deepmodeling/abacus-develop.git --depth 1 && cd abacus-develop && cmake -B build && cmake --build build -j`nproc` && cmake --install build && cd .. && rm -rf abacus-develop
7+
# If you have trouble cloning repo, replace "github.com" with "gitee.com".
8+
ENV OMPI_ALLOW_RUN_AS_ROOT=1 OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 OMPI_MCA_btl_vader_single_copy_mechanism=none
9+
CMD mpirun --use-hwthread-cpus abacus
10+
11+
# To run ABACUS built by this image with all available threads, execute `docker run -v <host>:<wd> -w <wd/input> abacus`.
12+
# Replace '<host>' with the path to all files(including pseudopotential files), '<wd>' with a path to working directory, and '<wd/input>' with the path to input folder(containing 'INPUT', 'STRU', etc.).
13+
# e.g. after clone the repo to `$HOME` and pulled this image, execute `docker run -v ~/abacus-develop/tests/integrate:/workspace -w /workspace/101_PW_15_f_pseudopots abacus`.
14+
# To run ABACUS with a given MPI process number, execute `docker run -v <host>:<wd> -w <wd/input> -it --entrypoint mpirun abacus -np <processes> abacus`. Note: the first "abacus" is the name of the image, the second "abacus" is the name of the executable file. Do not use '--cpus' flag of 'docker run' to specify the number of processes.
15+
16+
# To use this image as developing environment, execute `docker run -it --entrypoint /bin/bash abacus`.
17+
# Please refer to https://docs.docker.com/engine/reference/commandline/run/ for more details.

‎Dockerfile.cuda‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
FROM ghcr.io/deepmodeling/abacus-development-kit:gnu
22

3-
RUN wget https://developer.download.nvidia.com/compute/cuda/11.4.2/local_installers/cuda_11.4.2_470.57.02_linux.run --no-check-certificate --quiet \
4-
&& sh cuda_11.4.2_470.57.02_linux.run --toolkit --silent \
5-
&& rm cuda_11.4.2_470.57.02_linux.run
3+
RUN wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda_11.7.0_515.43.04_linux.run --no-check-certificate --quiet \
4+
&& sh cuda_11.7.0_515.43.04_linux.run --toolkit --silent \
5+
&& rm cuda_11.7.0_515.43.04_linux.run
66
ENV PATH=/usr/local/cuda/bin:$PATH

‎docs/CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ This file provides a guideline for it.
66
## Table of Contents
77

88
- [Got a question?](#got-a-question)
9-
- [Before getting started](#before-getting-started)
109
- [Structure of the package](#structure-of-the-package)
1110
- [Submitting an Issue](#submitting-an-issue)
1211
- [Comment Style for documentation](#comment-style-for-documentation)
@@ -23,6 +22,7 @@ If you would like to implement a new feature, please submit an issue with a prop
2322

2423
## Structure of the package
2524

25+
Please refer to [our instructions](./install.md) on how to installing ABACUS.
2626
The source code of ABACUS is based on several modules. Under the ABACUS root directory, there are the following folders:
2727

2828
- `cmake`: relevant files for finding required packages when compiling the code with cmake;

‎docs/features.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ equation. For PW basis, there are CG and Blocked Davidson methods for solving th
7474
equation for each basis.
7575

7676
- PW: ks_solver = ‘cg’ or ‘dav’
77-
- LCAO: ks_solver = ‘hpseps’ , ‘genelpa’ or ‘lapack’
77+
- LCAO: ks_solver = ‘hpseps’ , ‘genelpa’ , ‘scalapack_gvx’ or 'cusolver'
7878
- LCAO_in_PW: ks_solver = ‘lapack’
7979

8080
If you set ks_solver=‘hpseps’ for basis_type=‘pw’, the program will be stopped with an error

0 commit comments

Comments
 (0)