|
1 | 1 | # Continuous Integration in pySDC |
2 | 2 |
|
3 | | -Any commit in `pySDC` are tested within by GitHub continuous integration (CI). You can see in in the [action panel](https://github.com/Parallel-in-Time/pySDC/actions) the tests for each branches. |
4 | | -Those tests can be divided in two main categories : [code linting](#code-linting) and [code testing](#code-testing). |
| 3 | +Any commit in `pySDC` are tested by GitHub continuous integration (CI). You can see in in the [action panel](https://github.com/Parallel-in-Time/pySDC/actions) the tests for each branches. |
| 4 | +Those tests can be divided in three main categories : [code linting](#code-linting), [code testing](#code-testing) and [code coverage](#code-coverage). |
5 | 5 | Finally, the CI also build artifacts that are used to generate the documentation website (see http://parallel-in-time.org/pySDC/), more details given in the [documentation generation](#documentation-generation) section. |
6 | 6 |
|
7 | 7 | ## Code linting |
@@ -48,6 +48,62 @@ pytest -v pySDC/tests |
48 | 48 | > pytest -v pySDC/tests/test_nodes.py # only test nodes generation |
49 | 49 | > ``` |
50 | 50 |
|
| 51 | +## Code coverage |
| 52 | +
|
| 53 | +This stage allows to checks how much part of the `pySDC` code is tested by the previous stage. It is based on the [coverage](https://pypi.org/project/coverage/) library and currently applied to the following directories : |
| 54 | +
|
| 55 | +- `pySDC/core` |
| 56 | +- `pySDC/projects` |
| 57 | +- `pySDC/tutorial` |
| 58 | +
|
| 59 | +This analysis is done in parallel to the test each time a pull is done on any branch (main repository or fork). |
| 60 | +You can look at the current coverage report for the master branch [here](https://parallel-in-time.org/pySDC/coverage/index.html). |
| 61 | +
|
| 62 | +During developments, you can also run the coverage tests locally, using : |
| 63 | +
|
| 64 | +```bash |
| 65 | +echo "print('Loading sitecustomize.py...');import coverage;coverage.process_startup()" > sitecustomize.py |
| 66 | +coverage run -m pytest --continue-on-collection-errors -v --durations=0 pySDC/tests |
| 67 | +``` |
| 68 | +
|
| 69 | +> :bell: Note that this will run all `pySDC` tests while analyzing coverage, hence requires all packages installed for the [code testing stage](#code-testing). |
| 70 | +
|
| 71 | +Once the test are finished, you can collect and post-process coverage result : |
| 72 | + |
| 73 | +```bash |
| 74 | +coverage combine |
| 75 | +python -m coverage html |
| 76 | +``` |
| 77 | + |
| 78 | +This will generate the coverage report in a `htmlcov` folder, and you can open the `index.html` file within using your favorite browser. |
| 79 | + |
| 80 | +> :warning: Coverage can be lower if some tests fails (for instance, if you did not install all required python package to run all the tests). |
| 81 | +
|
| 82 | +### Coverage exceptions |
| 83 | + |
| 84 | +Some types of code lines will be ignored by the coverage analysis (_e.g_ lines starting with `raise`, ...), see the `[tool.coverage.report]` section in `pyproject.toml`. |
| 85 | +Part of code (functions, conditionaly, for loops, etc ...) can be ignored by coverage analysis using the `# pragma: no cover`, for instance |
| 86 | + |
| 87 | +```python |
| 88 | +# ... |
| 89 | +# code analyzed by coverage |
| 90 | +# ... |
| 91 | +# pragma: no cover |
| 92 | +if condition: |
| 93 | + # code ignored by coverage |
| 94 | +# ... |
| 95 | +# code analyzed by coverage |
| 96 | +# ... |
| 97 | +# pragma: no cover |
| 98 | +def function(): |
| 99 | + # all function code is ignored by coverage |
| 100 | +``` |
| 101 | + |
| 102 | +Accepted use of the `# pragma: no cover` are: |
| 103 | + |
| 104 | +1. Functions and code used for plotting |
| 105 | +2. Lines in one conditional preceding any `raise` statement |
| 106 | + |
51 | 107 | ## Documentation generation |
52 | 108 |
|
53 | 109 | Documentation is built using [sphinx](https://www.sphinx-doc.org/en/master/). |
|
0 commit comments