Skip to content

Commit 4e6bd57

Browse files
committed
TL: added contribution guidelines for coverage
1 parent 07db8c0 commit 4e6bd57

File tree

1 file changed

+58
-2
lines changed

1 file changed

+58
-2
lines changed

docs/contrib/02_continuous_integration.md

Lines changed: 58 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Continuous Integration in pySDC
22

3-
Any commit in `pySDC` are tested within by GitHub continuous integration (CI). You can see in in the [action panel](https://github.com/Parallel-in-Time/pySDC/actions) the tests for each branches.
4-
Those tests can be divided in two main categories : [code linting](#code-linting) and [code testing](#code-testing).
3+
Any commit in `pySDC` are tested by GitHub continuous integration (CI). You can see in in the [action panel](https://github.com/Parallel-in-Time/pySDC/actions) the tests for each branches.
4+
Those tests can be divided in three main categories : [code linting](#code-linting), [code testing](#code-testing) and [code coverage](#code-coverage).
55
Finally, the CI also build artifacts that are used to generate the documentation website (see http://parallel-in-time.org/pySDC/), more details given in the [documentation generation](#documentation-generation) section.
66

77
## Code linting
@@ -48,6 +48,62 @@ pytest -v pySDC/tests
4848
> pytest -v pySDC/tests/test_nodes.py # only test nodes generation
4949
> ```
5050
51+
## Code coverage
52+
53+
This stage allows to checks how much part of the `pySDC` code is tested by the previous stage. It is based on the [coverage](https://pypi.org/project/coverage/) library and currently applied to the following directories :
54+
55+
- `pySDC/core`
56+
- `pySDC/projects`
57+
- `pySDC/tutorial`
58+
59+
This analysis is done in parallel to the test each time a pull is done on any branch (main repository or fork).
60+
You can look at the current coverage report for the master branch [here](https://parallel-in-time.org/pySDC/coverage/index.html).
61+
62+
During developments, you can also run the coverage tests locally, using :
63+
64+
```bash
65+
echo "print('Loading sitecustomize.py...');import coverage;coverage.process_startup()" > sitecustomize.py
66+
coverage run -m pytest --continue-on-collection-errors -v --durations=0 pySDC/tests
67+
```
68+
69+
> :bell: Note that this will run all `pySDC` tests while analyzing coverage, hence requires all packages installed for the [code testing stage](#code-testing).
70+
71+
Once the test are finished, you can collect and post-process coverage result :
72+
73+
```bash
74+
coverage combine
75+
python -m coverage html
76+
```
77+
78+
This will generate the coverage report in a `htmlcov` folder, and you can open the `index.html` file within using your favorite browser.
79+
80+
> :warning: Coverage can be lower if some tests fails (for instance, if you did not install all required python package to run all the tests).
81+
82+
### Coverage exceptions
83+
84+
Some types of code lines will be ignored by the coverage analysis (_e.g_ lines starting with `raise`, ...), see the `[tool.coverage.report]` section in `pyproject.toml`.
85+
Part of code (functions, conditionaly, for loops, etc ...) can be ignored by coverage analysis using the `# pragma: no cover`, for instance
86+
87+
```python
88+
# ...
89+
# code analyzed by coverage
90+
# ...
91+
# pragma: no cover
92+
if condition:
93+
# code ignored by coverage
94+
# ...
95+
# code analyzed by coverage
96+
# ...
97+
# pragma: no cover
98+
def function():
99+
# all function code is ignored by coverage
100+
```
101+
102+
Accepted use of the `# pragma: no cover` are:
103+
104+
1. Functions and code used for plotting
105+
2. Lines in one conditional preceding any `raise` statement
106+
51107
## Documentation generation
52108

53109
Documentation is built using [sphinx](https://www.sphinx-doc.org/en/master/).

0 commit comments

Comments
 (0)