Skip to content

[#372] Enhance coverage script to generate branch coverage reports for workspace and individual components #1350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Afshankk wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[#372] Enhance coverage script to generate branch coverage reports for workspace and individual components #1350

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/reuse_coverage.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ jobs:
mode: "pre"

- name: Generate raw coverage results
run: ./internal/scripts/generate-cov-report.sh --generate
run: ./internal/scripts/generate-cov-report.sh --generate --branches

- name: Generate coverage results for html artifacts
run: ./internal/scripts/generate-cov-report.sh --html
run: ./internal/scripts/generate-cov-report.sh --html --branches

- name: Archive coverage-html artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # version: v6.0.0
Expand All @@ -66,7 +66,7 @@ jobs:
retention-days: 90

- name: Generate coverage reports for Codecov
run: ./internal/scripts/generate-cov-report.sh --lcov
run: ./internal/scripts/generate-cov-report.sh --lcov --branches

- name: Upload Rust coverage to Codecov
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # version: v5.5.2
Expand Down
4 changes: 3 additions & 1 deletion doc/release-notes/iceoryx2-unreleased.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,9 @@
conflicts when merging.
-->

* Example text [#1](https://github.com/eclipse-iceoryx/iceoryx2/issues/1)
* Enhance coverage script to generate branch coverage reports for workspace and
individual components
[#372](https://github.com/eclipse-iceoryx/iceoryx2/issues/372)

### New API features

Expand Down
124 changes: 103 additions & 21 deletions internal/scripts/generate-cov-report.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ REPORT=0
OVERVIEW=0
HTML=0
LCOV=0
COMPONENTS=()
SHOW_BRANCHES=0

dependency_check() {
which $1 1> /dev/null || { echo -e "${COLOR_RED}'${1}' not found. Aborting!${COLOR_OFF}"; exit 1; }
Expand All @@ -65,7 +67,19 @@ generate_full_profile() {

generate_rust_profile() {
set_rustc_flags
cargo test --workspace --all-targets -- --test-threads=1

if [[ ${#COMPONENTS[@]} -gt 0 ]]; then
# Test specific components
for COMPONENT in "${COMPONENTS[@]}"
do
echo -e "${COLOR_GREEN}Running tests for component: ${COMPONENT}${COLOR_OFF}"
cargo test -p "${COMPONENT}" --all-targets -- --test-threads=1
done
else
# Test entire workspace
echo -e "${COLOR_GREEN}Running tests for entire workspace${COLOR_OFF}"
cargo test --workspace --all-targets -- --test-threads=1
fi
}

generate_cmake_profile() {
Expand Down Expand Up @@ -110,6 +124,18 @@ show_overview() {
local FILES=$(find ./target/debug/deps/ -type f -executable)
CMD="llvm-cov report --use-color --ignore-filename-regex='/.cargo/registry' --instr-profile=./${RUST_COV_DIR}/json5format.profdata"

if [[ $SHOW_BRANCHES == "1" ]]; then
CMD="$CMD --show-branch-summary"
fi

# Filter by component if specified
if [[ ${#COMPONENTS[@]} -gt 0 ]]; then
for COMPONENT in "${COMPONENTS[@]}"
do
FILES=$(echo "$FILES" | grep "$COMPONENT")
done
fi

for FILE in $FILES
do
CMD="$CMD --object $FILE"
Expand All @@ -125,7 +151,19 @@ show_report() {
merge_report

local FILES=$(find ./target/debug/deps/ -type f -executable)
CMD="llvm-cov report --use-color --ignore-filename-regex='/.cargo/registry' --instr-profile=./${RUST_COV_DIR}/json5format.profdata"
CMD="llvm-cov show --use-color --ignore-filename-regex='/.cargo/registry' --instr-profile=./${RUST_COV_DIR}/json5format.profdata"

if [[ $SHOW_BRANCHES == "1" ]]; then
CMD="$CMD --show-branches=count"
fi

# Filter by component if specified
if [[ ${#COMPONENTS[@]} -gt 0 ]]; then
for COMPONENT in "${COMPONENTS[@]}"
do
FILES=$(echo "$FILES" | grep "$COMPONENT")
done
fi

for FILE in $FILES
do
Expand All @@ -146,45 +184,72 @@ generate_report() {

mkdir -p ./${COVERAGE_OUT_DIR}

# Build grcov ignore list based on components
local GRCOV_IGNORES="--ignore '*iceoryx2-cli*' --ignore '*iceoryx2-ffi*' --ignore '*build.rs' --ignore '*tests*' --ignore '*testing*' --ignore '*component-tests*' --ignore '*examples*' --ignore '*benchmarks*' --ignore '*target*' --ignore '*.cargo*'"

# If specific components are requested, only include those
if [[ ${#COMPONENTS[@]} -gt 0 ]]; then
echo -e "${COLOR_GREEN}Generating coverage for components: ${COMPONENTS[*]}${COLOR_OFF}"
GRCOV_IGNORES=""
# Build a pattern to only include specified components
for COMPONENT in "${COMPONENTS[@]}"
do
# Don't ignore the specified component
GRCOV_IGNORES="$GRCOV_IGNORES --ignore '*iceoryx2*' --ignore '!*${COMPONENT}*'"
done
GRCOV_IGNORES="$GRCOV_IGNORES --ignore '*build.rs' --ignore '*tests*' --ignore '*testing*' --ignore '*component-tests*' --ignore '*examples*' --ignore '*benchmarks*' --ignore '*target*' --ignore '*.cargo*'"
fi

# Generate Coverage files for Rust Code
grcov \
eval grcov \
**/${LLVM_PROFILE_PATH} \
**/**/${LLVM_PROFILE_PATH} \
--binary-path target/debug \
--source-dir . \
--output-type ${OUTPUT_TYPE} \
--branch \
--ignore-not-existing \
--ignore "*iceoryx2-cli*" \
--ignore "*iceoryx2-ffi*" \
--ignore "*build.rs" \
--ignore "*tests*" \
--ignore "*testing*" \
--ignore "*component-tests*" \
--ignore "*examples*" \
--ignore "*benchmarks*" \
--ignore "*target*" \
--ignore "*.cargo*" \
${GRCOV_IGNORES} \
--llvm-path ${LLVM_PATH} \
--output-path ${GRCOV_OUTPUT_PATH}

# Generate Coverage files for C++ Code
# We use here https://github.com/gcovr/gcovr to handle the generated files by gcov and can be installed with `pip install gcovr`
# License: https://github.com/gcovr/gcovr/blob/main/LICENSE.txt
gcovr ${GCOVR_OUTPUT_PATH} ${CMAKE_COV_DIR} -e '/.*/_deps/' -e '/.*/tests/' -e '/.*/testing/' -e '/.*/component-tests/'
local GCOVR_BRANCH_FLAG=""
if [[ $SHOW_BRANCHES == "1" ]]; then
GCOVR_BRANCH_FLAG="--branches"
fi
Comment on lines +220 to +222
Copy link
Contributor

@dkroenke dkroenke Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this flag has any effect because we see the branch coverage already in the existing report for the C++ Code and this flag is in gcovr marked as deprecated and affects only the txt output.
When comparing the reports where the flag is enabled/disabled there seems to be no difference.


gcovr ${GCOVR_OUTPUT_PATH} ${GCOVR_BRANCH_FLAG} ${CMAKE_COV_DIR} -e '/.*/_deps/' -e '/.*/tests/' -e '/.*/testing/' -e '/.*/component-tests/'

}

show_help() {
echo "$0 [OPTIONS]"
echo
echo "-c|--clean - cleanup all reports"
echo "-g|--generate - generate coverage report"
echo "-o|--overview - show overview of coverage report"
echo "-r|--report - show detailed report"
echo "-l|--lcov - creates lcov report"
echo "-t|--html - creates html report"
echo "-f|--full - generate coverage report and create html and lcov"
echo "OPTIONS:"
echo " -c|--clean - cleanup all reports"
echo " -g|--generate - generate coverage report"
echo " -o|--overview - show overview of coverage report"
echo " -r|--report - show detailed report"
echo " -l|--lcov - creates lcov report"
echo " -t|--html - creates html report"
echo " -f|--full - generate coverage report and create html and lcov"
echo " -b|--branches - include branch coverage in reports"
Copy link
Contributor

@dkroenke dkroenke Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The branch coverage can be enabled by default so that the additional parameter here is not needed.

echo " --component <name> - generate coverage for specific component(s)"
echo " (can be specified multiple times)"
echo " Examples: iceoryx2, iceoryx2-cal, iceoryx2-bb-container"
echo
echo "EXAMPLES:"
echo " Generate full coverage with branch information:"
echo " $0 --full --branches"
echo
echo " Generate coverage for specific component:"
echo " $0 --generate --html --component iceoryx2-cal"
echo
echo " Generate coverage for multiple components:"
echo " $0 --generate --html --component iceoryx2 --component iceoryx2-cal"
Copy link
Contributor

@dkroenke dkroenke Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this command and it generates an empty report for the Rust code. The target folder was cleaned up before. It could be maybe related to the handling of GRCOV_IGNORES in the generate_report() function.
The Rust build and test respects the component setting correctly.

echo
exit 1
}
Expand Down Expand Up @@ -225,6 +290,18 @@ while [[ $# -gt 0 ]]; do
HTML=1
shift
;;
-b|--branches)
SHOW_BRANCHES=1
shift
;;
--component)
if [[ -z "$2" ]] || [[ "$2" == -* ]]; then
echo -e "${COLOR_RED}Error: --component requires a component name${COLOR_OFF}"
exit 1
fi
COMPONENTS+=("$2")
shift 2
;;
*)
show_help
;;
Expand Down Expand Up @@ -255,6 +332,11 @@ fi
if [[ $HTML == "1" ]]; then
mkdir -p $COVERAGE_OUT_DIR/html/cpp
generate_report "html" "${COVERAGE_OUT_DIR}/html/rust" "--html-details ${COVERAGE_OUT_DIR}/html/cpp/index.html"

if [[ ${#COMPONENTS[@]} -gt 0 ]]; then
echo -e "${COLOR_GREEN}Coverage report generated for components: ${COMPONENTS[*]}${COLOR_OFF}"
fi

echo "The Report for Rust Code in iceoryx2 is located at: ${COVERAGE_OUT_DIR}/html/rust/index.html"
echo "The Report for C++ Code in iceoryx2 is located at: ${COVERAGE_OUT_DIR}/html/cpp/index.html"
fi
Loading