Skip to content

Commit 35ba8ca

Browse files
author
Diptorup Deb
authored
Merge pull request #459 from diptorupd/feature/coverage_reports
Generate coverage data and publish them to coveralls.io
2 parents da988f9 + a98d4f9 commit 35ba8ca

File tree

4 files changed

+167
-46
lines changed

4 files changed

+167
-46
lines changed

‎.github/workflows/generate-coverage.yaml

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
name: Generate coverage data
2+
on:
3+
pull_request:
4+
push:
5+
branches: [master]
6+
7+
jobs:
8+
generate-coverage:
9+
name: Generate coverage and push to Coveralls.io
10+
runs-on: ubuntu-20.04
11+
12+
env:
13+
ONEAPI_ROOT: /opt/intel/oneapi
14+
GTEST_ROOT: /home/runner/work/googletest-release-1.10.0/install
15+
16+
steps:
17+
- name: Cancel Previous Runs
18+
uses: styfle/[email protected]
19+
with:
20+
access_token: ${{ github.token }}
21+
22+
- name: Add Intel repository
23+
run: |
24+
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2023.PUB
25+
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS-2023.PUB
26+
rm GPG-PUB-KEY-INTEL-SW-PRODUCTS-2023.PUB
27+
sudo add-apt-repository "deb https://apt.repos.intel.com/oneapi all main"
28+
sudo apt-get update
29+
30+
- name: Install Intel OneAPI
31+
run: |
32+
sudo apt-get install intel-oneapi-dpcpp-cpp-compiler
33+
sudo apt-get install intel-oneapi-tbb
34+
35+
- name: Install CMake
36+
run: |
37+
sudo apt-get install cmake
38+
39+
- name: Setup Python
40+
uses: actions/setup-python@v2
41+
with:
42+
python-version: '3.8'
43+
architecture: x64
44+
45+
- name: Cache Gtest
46+
id: cache-gtest
47+
uses: actions/cache@v2
48+
with:
49+
path: |
50+
/home/runner/work/googletest-release-1.10.0/install
51+
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('/home/runner/work/googletest-release-1.10.0/install/include/gtest/*') }}
52+
restore-keys: |
53+
${{ runner.os }}-build-${{ env.cache-name }}-
54+
${{ runner.os }}-build-
55+
${{ runner.os }}-
56+
57+
- name: Install Gtest
58+
if: steps.cache-gtest.outputs.cache-hit != 'true'
59+
shell: bash -l {0}
60+
run: |
61+
cd /home/runner/work
62+
wget https://github.com/google/googletest/archive/refs/tags/release-1.10.0.tar.gz
63+
tar xf release-1.10.0.tar.gz
64+
cd googletest-release-1.10.0
65+
mkdir build
66+
cd build
67+
cmake .. -DCMAKE_INSTALL_PREFIX=/home/runner/work/googletest-release-1.10.0/install
68+
make && make install
69+
70+
- name: Checkout repo
71+
uses: actions/checkout@v2
72+
with:
73+
fetch-depth: 0
74+
75+
- name: Install Lcov
76+
run: |
77+
sudo apt-get install lcov
78+
79+
- name: Install dpctl dependencies
80+
shell: bash -l {0}
81+
run: |
82+
pip install numpy cython setuptools pytest pytest-cov coverage
83+
84+
- name: Build dpctl with coverage
85+
shell: bash -l {0}
86+
run: |
87+
source /opt/intel/oneapi/setvars.sh
88+
python setup.py develop --coverage=True
89+
python -c "import dpctl; print(dpctl.__version__); dpctl.lsplatform()"
90+
pytest -q -ra --disable-warnings --cov dpctl --cov-report term-missing --pyargs dpctl -vv
91+
92+
- name: Install coverall dependencies
93+
shell: bash -l {0}
94+
run: |
95+
sudo gem install coveralls-lcov
96+
pip install coveralls
97+
98+
- name: Upload coverage data to coveralls.io
99+
run: |
100+
coveralls-lcov -v -n build_cmake/tests/dpctl.lcov > dpctl-c-api-coverage.json
101+
coveralls --service=github --merge=dpctl-c-api-coverage.json
102+
env:
103+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
104+
COVERALLS_PARALLEL: true
105+
106+
coveralls:
107+
name: Indicate completion to coveralls.io
108+
needs: generate-coverage
109+
runs-on: ubuntu-latest
110+
container: python:3-slim
111+
steps:
112+
- name: Finished
113+
run: |
114+
pip3 install --upgrade coveralls
115+
coveralls --finish
116+
env:
117+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎CONTRIBUTING.md

Lines changed: 48 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,20 @@ Install: `conda install clang-tools`
1111
- Revision: `10.0.1`
1212
- See the default configuration used by dpctl in `.clang-format`.
1313

14-
Run before each commit: `clang-format -style=file -i dpctl-capi/include/*.h dpctl-capi/include/Support/*.h dpctl-capi/source/*.cpp dpctl-capi/tests/*.cpp dpctl-capi/helper/include/*.h dpctl-capi/helper/source/*.cpp`
14+
Run before each commit:
15+
16+
```bash
17+
clang-format -style=file -i \
18+
dpctl-capi/include/*.h \
19+
dpctl-capi/include/Support/*.h \
20+
dpctl-capi/source/*.cpp \
21+
dpctl-capi/tests/*.cpp \
22+
dpctl-capi/helper/include/*.h \
23+
dpctl-capi/helper/source/*.cpp
24+
```
25+
26+
> **_NOTE:_** A much simpler option is to use `pre-commit` and the
27+
> `clang-format` hook that we provide.
1528
1629
### Python code style
1730

@@ -121,61 +134,51 @@ Run before each commit: `bandit -r dpctl -lll`
121134

122135
## Code Coverage
123136

124-
Implement python, cython and c++ file coverage using `coverage` and `llvm-cov`
125-
packages on Linux.
126-
127-
### Using Code Coverage
128-
129-
You need to install additional packages and add an environment variable to
130-
cover:
131-
- conda install cmake
132-
- conda install coverage
133-
- conda install conda-forge::lcov
134-
- conda install conda-forge::gtest
135-
- export CODE_COVERAGE=ON
136-
137-
CODE_COVERAGE allows you to build a debug version of dpctl and enable string
138-
tracing, which allows you to analyze strings to create a coverage report.
139-
It was added for the convenience of configuring the CI in the future.
140-
141-
Installing the dpctl package:
142-
- python setup.py develop
137+
Code coverage for both C and Python sources in dpctl is generated for each
138+
pull request (PR). A PR cannot be merged if it leads to a drop in the code
139+
coverage by more than five percentage points. *Ergo, do not forget to write
140+
unit tests for your changes.* To check the code coverage for your code, follow
141+
these steps:
143142

144-
It is important that there are no files of the old build in the folder.
145-
Use `git clean -xdf` to clean up the working tree.
143+
1. Install dependencies for C/C++ source.
146144

147-
The coverage report will appear during the build in the console. This report
148-
contains information about c++ file coverage.
149-
The `dpctl-c-api-coverage` folder will appear in the root folder after
150-
installation. The folder contains a report on the coverage of c++ files in html
151-
format.
145+
For C/C++ source we require `lcov`, `llvm-cov`, and `llvm-profdata`. Note
146+
that `llvm-cov` and `llvm-profdata` should be version 11.0 or higher. If you
147+
have multiple `llvm-cov` tools installed, most likely because you have
148+
multiple llvm installations, you should set the `LLVM_TOOLS_HOME`
149+
environment variable to make sure the correct one is used to generate
150+
coverage.
152151

153-
You need to run tests to cover the cython and python files:
154-
- coverage run -m unittest dpctl.tests
152+
2. Install dependencies for Python sources.
155153

156-
The required flags for the command coverage run are contained in the file
157-
`.coveragerc`.
154+
To generate the coverage data for dpctl's Python sources, you only need to
155+
install `coverage`.
158156

159-
The simplest reporting is a textual summary produced with report:
160-
- coverage report
157+
3. Build dpctl with code coverage support.
161158

162-
For each module executed, the report shows the count of executable statements,
163-
the number of those statements missed, and the resulting coverage, expressed as
164-
a percentage.
159+
```bash
160+
python setup.py develop --coverage=True
161+
pytest -q -ra --disable-warnings --cov dpctl --cov-report term-missing --pyargs dpctl -vv
162+
coverage html
163+
```
165164

166-
The `-m` flag also shows the line numbers of missing statements:
167-
- coverage report -m
165+
Note that code coverage builds the C sources with debug symbols. For this
166+
reason, the coverage flag is only available with the `develop` mode of
167+
`setup.py`.
168168

169-
To create an annotated HTML listings with coverage results:
170-
- coverage html
169+
The coverage results for the C and Python sources will be printed to the
170+
terminal during the build (C API) and during the pytest execution (Python).
171+
The detailed coverage reports for the C API is saved to the
172+
`dpctl-c-api-coverage` directory. The Python coverage reports are saved to
173+
the `htmlcov` directory.
171174

172-
The `htmlcov` folder will appear in the root folder of the project. It contains
173-
reports on the coverage of python and cython files in html format.
175+
The coverage data for every PR is also available online at
176+
[coveralls.io](https://coveralls.io/github/IntelPython/dpctl).
174177

175-
Erase previously collected coverage data:
176-
- coverage erase
178+
> **_NOTE:_** Run `git clean -xdf` to clean up the working tree before running
179+
> a fresh build with code coverage data generation.
177180

178-
### Error in the build process
181+
## Error in the build process
179182

180183
An error occurs during the dcptl build with the CODE_COVERAGE environment
181184
variable:

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
22
[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
3+
[![Coverage Status](https://coveralls.io/repos/github/IntelPython/dpctl/badge.svg?branch=master)](https://coveralls.io/github/IntelPython/dpctl?branch=master)
34

45
What?
56
====

‎scripts/build_for_develop.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@ set +xe
33

44
python setup.py clean --all
55
python setup.py develop --coverage=True
6-
pytest -q -ra --disable-warnings --cov dpctl --cov-report term-missing --pyargs dpctl -vv --cov-config=.coveragerc
6+
pytest -q -ra --disable-warnings --cov dpctl --cov-report term-missing --pyargs dpctl -vv

0 commit comments

Comments
 (0)