Skip to content

Commit c6429ad

Browse files
abussymfherbstTechnici4n
authored
Refactoring of tests (#1143)
--------- Co-authored-by: Michael F. Herbst <[email protected]> Co-authored-by: Bruno Ploumhans <[email protected]>
1 parent 477346f commit c6429ad

20 files changed

+97
-64
lines changed

‎.github/workflows/ci.yaml‎

Lines changed: 29 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,18 +14,39 @@ concurrency:
1414
cancel-in-progress: ${{ startsWith(github.ref, 'refs/pull/') }}
1515

1616
jobs:
17+
build_matrix:
18+
name: Build Test Matrix
19+
runs-on: ubuntu-latest
20+
outputs:
21+
matrix: ${{ steps.set-matrix.outputs.matrix }}
22+
steps:
23+
- run: |
24+
cat > matrix.yaml <<- EOM
25+
- {mode: stable, arch: x64, os: ubuntu-latest, payload: minimal }
26+
- {mode: stable, arch: x64, os: ubuntu-latest, payload: nominimal-noslow }
27+
- {mode: stable, arch: x64, os: ubuntu-latest, payload: example }
28+
- {mode: stable, arch: aarch64, os: macOS-latest, payload: minimal }
29+
- {mode: stable, arch: x64, os: ubuntu-latest, payload: noslow-mpi }
30+
- {mode: latest, arch: x64, os: ubuntu-latest, payload: minimal }
31+
EOM
32+
# Windows tests are disabled for PR builds
33+
- if: ${{ github.event_name != 'pull_request' }}
34+
run: |
35+
cat >> matrix.yaml <<- EOM
36+
- {mode: latest, arch: x64, os: windows-latest, payload: minimal }
37+
EOM
38+
- name: Convert to JSON
39+
run: |
40+
python3 -c 'import sys, yaml, json; json.dump(yaml.safe_load(sys.stdin), sys.stdout)' < matrix.yaml > matrix.json
41+
- id: set-matrix
42+
run: echo "matrix={\"include\":$(cat matrix.json)}" >> $GITHUB_OUTPUT
1743
test:
1844
name: Julia ${{ matrix.mode }} - ${{ matrix.os }} - ${{ matrix.payload }}
45+
needs: build_matrix
1946
runs-on: ${{ matrix.os }}
2047
strategy:
2148
fail-fast: false
22-
matrix:
23-
include:
24-
- {mode: stable, os: ubuntu-latest, payload: noslow-example }
25-
- {mode: stable, os: macOS-latest, payload: noslow }
26-
- {mode: stable, os: windows-latest, payload: noslow }
27-
- {mode: stable, os: ubuntu-latest, payload: noslow-mpi }
28-
- {mode: latest, os: ubuntu-latest, payload: noslow }
49+
matrix: ${{ fromJson(needs.build_matrix.outputs.matrix) }}
2950
env:
3051
GKS_ENCODING: utf8
3152
GKSwstype: 100 # Needed for Plots-related tests
@@ -39,7 +60,7 @@ jobs:
3960
uses: julia-actions/setup-julia@v2
4061
with:
4162
version: '1.10'
42-
arch: x64
63+
arch: ${{ matrix.arch }}
4364
if: ${{ matrix.mode == 'stable' }}
4465
- name: Setup Julia latest (pre-releases included)
4566
uses: julia-actions/setup-julia@v2

‎docs/src/developer/testsystem.md‎

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,18 +10,25 @@ the tests.
1010

1111
## Running selective tests
1212
### Selecting by tags
13-
To only run core functionality tests (those which are tagged `:core`) one can simply run
13+
To only run a minimal set of tests designed to ensure DFTK functionality (tests tagged with `:minimal`),
14+
on can simply run
1415
```julia
1516
using Pkg
16-
Pkg.test("DFTK"; test_args = ["noall", "core"])
17+
Pkg.test("DFTK"; test_args = ["minimal"])
1718
```
18-
where `"noall"` disables the standard test suite and `"core"` exactly selects the core tests.
19-
Similarly
19+
By default, all tests are run. Specifying any subset implicitly turns off all tests not
20+
tagged accordingly. Multiple tags can be specified at once. For example,
2021
```julia
2122
using Pkg
22-
Pkg.test("DFTK"; test_args = ["noall", "core", "atomsbase"])
23+
Pkg.test("DFTK"; test_args = ["forces", "example"])
2324
```
24-
runs both `:core` and `:atomsbase` tests.
25+
will test forces and run the examples. It is also possible to disable certain tests:
26+
```julia
27+
using Pkg
28+
Pkg.test("DFTK"; test_args = ["noslow"])
29+
```
30+
will ignore any test tagged as `:slow`. Finally, parallel tests can be run by passing `"mpi"` to
31+
the `test_args` keyword argument. GPU tests are triggered with the `"gpu"` tag.
2532

2633
### Selecting by file name
2734
This works by directly instantiating the test environment and triggering

‎test/compute_bands.jl‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ end
115115
@test kinter[8] == [0.5]
116116
end
117117

118-
@testitem "Compute bands for silicon" tags=[:dont_test_mpi] setup=[TestCases] begin
118+
@testitem "Compute bands for silicon" tags=[:dont_test_mpi, :minimal] setup=[TestCases] begin
119119
using DFTK
120120
using Brillouin: interpolate
121121
testcase = TestCases.silicon
@@ -217,7 +217,7 @@ end
217217
end
218218
end
219219

220-
@testitem "compute_bands for meta-GGA" tags=[:dont_test_mpi] setup=[TestCases] begin
220+
@testitem "compute_bands for meta-GGA" tags=[:dont_test_mpi, :minimal] setup=[TestCases] begin
221221
# This triggers a bug reported previously, see
222222
# https://github.com/JuliaMolSim/DFTK.jl/issues/1065
223223
using DFTK

‎test/external_potential.jl‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
@testitem "External potential from Fourier coefficients" #=
2-
=# tags=[:core, :dont_test_mpi] begin
2+
=# tags=[:dont_test_mpi] begin
33
using DFTK
44

55
lattice = [[10 0 0.]; [0 0 0]; [0 0 0]]

‎test/forces.jl‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@
100100
end
101101
end
102102

103-
@testitem "Forces term-wise TiO2 (GTH)" setup=[TestForces] tags=[:forces] begin
103+
@testitem "Forces term-wise TiO2 (GTH)" setup=[TestForces] tags=[:forces, :minimal] begin
104104
# Test HF forces on non-symmetric multi-species structure using analytical pseudos
105105
using AtomsIO
106106
using PseudoPotentialData
@@ -110,7 +110,7 @@ end
110110
pseudopotentials, mixing=DielectricMixing(εr=10))
111111
end
112112

113-
@testitem "Forces term-wise TiO2 (UPF)" setup=[TestForces] tags=[:forces] begin
113+
@testitem "Forces term-wise TiO2 (UPF)" setup=[TestForces] tags=[:forces, :minimal] begin
114114
# Test HF forces on non-symmetric multi-species structure with NLCC
115115
using AtomsIO
116116
system = load_system("structures/tio2_stretched.extxyz")

‎test/forwarddiff.jl‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
@testitem "Force derivatives using ForwardDiff" #=
2-
=# tags=[:dont_test_mpi] setup=[TestCases] begin
2+
=# tags=[:dont_test_mpi, :minimal] setup=[TestCases] begin
33
using DFTK
44
using ForwardDiff
55
using LinearAlgebra
@@ -68,7 +68,7 @@
6868
end
6969

7070
@testitem "Anisotropic strain sensitivity using ForwardDiff" #=
71-
=# tags=[:dont_test_mpi] setup=[TestCases] begin
71+
=# tags=[:dont_test_mpi, :minimal] setup=[TestCases] begin
7272
using DFTK
7373
using ForwardDiff
7474
using LinearAlgebra
@@ -118,7 +118,7 @@ end
118118
end
119119

120120
@testitem "scfres PSP sensitivity using ForwardDiff" #=
121-
=# tags=[:dont_test_mpi] setup=[TestCases] begin
121+
=# tags=[:dont_test_mpi, :minimal] setup=[TestCases] begin
122122
using DFTK
123123
using ForwardDiff
124124
using LinearAlgebra
@@ -160,7 +160,7 @@ end
160160
end
161161

162162
@testitem "Functional force sensitivity using ForwardDiff" #=
163-
=# tags=[:dont_test_mpi] setup=[TestCases] begin
163+
=# tags=[:dont_test_mpi, :minimal] setup=[TestCases] begin
164164
using DFTK
165165
using ForwardDiff
166166
using LinearAlgebra
@@ -191,7 +191,7 @@ end
191191
@test norm(derivative_ε - derivative_fd) < 1e-4
192192
end
193193

194-
@testitem "Derivative of complex function" tags=[:dont_test_mpi] begin
194+
@testitem "Derivative of complex function" tags=[:dont_test_mpi, :minimal] begin
195195
using DFTK
196196
using ForwardDiff
197197
using LinearAlgebra
@@ -207,7 +207,7 @@ end
207207
@test norm(fd1 - fd2) < 1e-8
208208
end
209209

210-
@testitem "Higher derivatives of Fermi-Dirac occupation" tags=[:dont_test_mpi] begin
210+
@testitem "Higher derivatives of Fermi-Dirac occupation" tags=[:dont_test_mpi, :minimal] begin
211211
using DFTK
212212
using ForwardDiff
213213

@@ -232,7 +232,7 @@ end
232232
end
233233
end
234234

235-
@testitem "LocalNonlinearity sensitivity using ForwardDiff" tags=[:dont_test_mpi] begin
235+
@testitem "LocalNonlinearity sensitivity using ForwardDiff" tags=[:dont_test_mpi, :minimal] begin
236236
using DFTK
237237
using ForwardDiff
238238
using LinearAlgebra

‎test/helium_all_electron.jl‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
@testitem "Helium all electron" tags=[:minimal, :core, :dont_test_mpi] begin
1+
@testitem "Helium all electron" tags=[:minimal, :dont_test_mpi] begin
22
using DFTK
33
using LinearAlgebra
44

‎test/iron_lda.jl‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,6 @@ end
4848
end
4949

5050

51-
@testitem "Iron LDA (Float64)" tags=[:core] setup=[RunSCF, TestCases, IronLDA] begin
51+
@testitem "Iron LDA (Float64)" tags=[:minimal] setup=[RunSCF, TestCases, IronLDA] begin
5252
IronLDA.run_iron_lda(Float64; test_tol=5e-6, scf_ene_tol=1e-11)
5353
end

‎test/occupation.jl‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ end
9595
end
9696

9797
@testitem "Smearing for a simple metal" #=
98-
=# tags=[:dont_test_mpi] setup=[Occupation, TestCases] begin
98+
=# tags=[:dont_test_mpi, :minimal] setup=[Occupation, TestCases] begin
9999
using DFTK
100100
using Logging
101101
(; silicon, magnesium) = TestCases.all_testcases
@@ -139,7 +139,7 @@ end
139139
end
140140

141141
@testitem "Fermi level finding for smearing multiple εF" #=
142-
=# tags=[:dont_test_mpi] setup=[Occupation, TestCases] begin
142+
=# tags=[:dont_test_mpi, :minimal] setup=[Occupation, TestCases] begin
143143
using DFTK
144144
using Logging
145145
iron_bcc = TestCases.iron_bcc

‎test/runtests.jl‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,15 @@
11
#
22
# This test suite supports test arguments. For example:
3-
# Pkg.test("DFTK"; test_args = ["core"])
3+
# Pkg.test("DFTK"; test_args = ["minimal"])
44
# only runs the "fast" tests (i.e. not the expensive ones)
55
# Pkg.test("DFTK"; test_args = ["gpu"])
66
# runs only the tests tagged as "gpu" and
7-
# Pkg.test("DFTK"; test_args = ["example", "all"])
8-
# runs all tests plus the "example" tests.
9-
#
7+
# Pkg.test("DFTK"; test_args = ["mpi"])
8+
# runs only MPI parallel tests (all except those with :dont_test_mpi tag)
9+
# Pkg.test("DFTK"; test_args = ["mpi", "minimal"])
10+
# runs only MPI parallel tests from the :minimal subset
11+
# Pkg.test("DFTK"; test_args = ["noslow"])
12+
# runs all tests except those tagges as :slow
1013

1114
using MPI
1215
include("runtests_parser.jl")

0 commit comments

Comments
 (0)