coverage report of lcov1.14 is not correct

[for2years]

Environment

$cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.5 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

$lcov --version
lcov: LCOV version 1.14

I installed lcov by apt install lcov.

Problem Description

I collect coverage report by the following commands:

WORKSPACE=$(realpath ".")
find . -name "*.gcda" -print0 | xargs -0 rm -f

lcov --rc lcov_branch_coverage=1 --directory build/.objs --base-directory "${WORKSPACE}" --include "${WORKSPACE}/zmath/*" --exclude "${WORKSPACE}/zmath/*_test.cc" --gcov-tool "${BASE_DIR}/llvm-gcov.sh" --capture -o coverage.info
genhtml -s --rc lcov_branch_coverage=1 coverage.info --branch-coverage --output-directory coverage

(cd coverage && python3 -m http.server 17777)

but the result is so confused:

[hartwork]

@for2years LCOV 1.14 is from the stone age — "Maintenance & Security Support ends in 1 month and 2 weeks
(02 Apr 2025)" and I just dropped support for Ubuntu 20.04 in pull request #391. Could you try reproduce the issue with latest Git master please? A lot has happened since 1.14. Thank you! 🙏

[for2years]

Could you try reproduce the issue with latest Git master please?

I install the lcov with latest git master:

git clone https://github.com/linux-test-project/lcov.git
cd lcov
make install

then I meet this error:

$ lcov --version
Can't locate Capture/Tiny.pm in @INC (you may need to install the Capture::Tiny module) (@INC contains: /usr/local/lib/lcov /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/local/lib/lcov/lcovutil.pm line 16.
BEGIN failed--compilation aborted at /usr/local/lib/lcov/lcovutil.pm line 16.
Compilation failed in require at /usr/local/bin/lcov line 101.
BEGIN failed--compilation aborted at /usr/local/bin/lcov line 101.

try to fix this error:

apt install libcapture-tiny-perl libdatetime-perl

Looks like it solved:

$lcov --version
lcov: LCOV version 2.4-beta

But I encountered a new problem:

Capturing coverage data from build/.objs
geninfo cmd: '/usr/local/bin/geninfo build/.objs --toolname lcov --output-filename coverage.info --base-directory . --ignore-errors gcov --ignore-errors gcov --ignore-errors unsupported --ignore-errors unsupported --ignore-errors source --branch-coverage --rc branch_coverage=1 --gcov-tool /zelos/zmath/script/llvm-gcov.sh'
Found LLVM gcov version 14.0.6, which emulates gcov version 4.2.0
Using intermediate gcov format
Recording 'internal' directories:
        /zelos/zmath/build/.objs
        build/.objs
        /zelos/zmath
Writing temporary data to /tmp/geninfo_datxEH1
Scanning build/.objs for .gcda files ...
Found 73 data files in build/.objs
using: chunkSize: 1, nchunks:73, intervalLength:3
Finished processing 73 GCDA files
Apply filtering..
lcov: WARNING: (source) unable to open /zelos/zmath/build/.objs/zmath.discretized_qp.discretized_qp_solver_test/linux/x86_64/coverage/zmath/discretized_qp/__cpp_discretized_qp_solver_test.cc.cc: No such file or directory
        (use "lcov --ignore-errors source,source ..." to suppress this warning)
lcov: WARNING: (source) unable to open /zelos/zmath/build/.objs/zmath.math/linux/x86_64/coverage/zmath/math/__cpp_circle2d.cc.cc: No such file or directory
lcov: ERROR: (inconsistent) "/usr/local/include/Eigen/src/Core/CwiseUnaryOp.h":94:  function _ZN5Eigen16CwiseUnaryOpImplINS_8internal14scalar_abs2_opIdEEKNS_6MatrixIdLi3ELi1ELi0ELi3ELi1EEENS_5DenseEEC2Ev found on line but no corresponding 'line' coverage data point.  Cannot derive function end line.  See lcovrc man entry for 'derive_function_end_line'.
        (use "lcov --ignore-errors inconsistent ..." to bypass this error)
Message summary:
  1 error message:
    inconsistent: 1
  2 warning messages:
    source: 2
  3 ignore messages:
    gcov: 2
    unsupported: 1

[for2years]

can I ignore all errors?
now I use the following commands like:

lcov --ignore-errors gcov,unsupported,range,inconsistent,source,format    ...

and I also want to know is it a good behavior to ignore errors?

[hartwork]

@for2years thanks for the update. I'm not a maintainer here but if I was I would suggest to create a dedicated issue with a minimal reproducer — a shell session or a Dockerfile — for the new symptoms with master, and then close this one here as obsolete. For the new symptoms, I'm not deep enough into LCOV to understand the case. We'll probably have to wait for #390 (comment) .

[henry2cox]

can I ignore all errors? now I use the following commands like:

lcov --ignore-errors gcov,unsupported,range,inconsistent,source,format ...

If you really want to ignore all errors, then you can use:
lcov --keep-going ...
The tool will return a non-zero exit code if any ignorable errors occurred- but won't stop when it hits an error.

and I also want to know is it a good behavior to ignore errors?

Depends on your motivation.

• if you are in a high reliability environment/have high quality requirements: then you probably want to review the messages to understand the root causes - and fix them, if possible. For example, missing sources may be due to path or permission issues. Inconsistencies may be caused by version mismatches, tool bugs, etc.
  You don't want to blindly ignore problems which may be telling you that your results are unreliable or incorrect.

• if you are only looking for a number and don't intend to act on the result (e.g., to add more tests, refactor the code, etc.) - then ignoring everything reduces the amount of work you have to do - so why not minimize your effort?

If you are closer to the first environment than the second: it is likely that some warning/error messages remain once you have addressed all the issues to your satisfaction (e.g., because yacc/lex are numbering things incorrectly).
In that case: there is is a feature to tell the tool how many messages you expect to see - and it will give you an error if the count comes up differently. See the man pages for details.

With respect to you original issue/original question:

• see What if I am using clang/llvm rather than gcc? #234
• LLVM has a number of bugs/issues that you will see in both the "gcov path" and in the "profdata path".
  Newer versions are generally better than older so upgrading your toolchain is a good idea.
  My experience is that the profdata path is generally more reliable after about LLVM/15 (but plenty of data consistency issues remain even in the most recent versions).
  You need to be using LLVM/18.1 or newer if you want to use MC/DC metrics.

Please try your testcases again with a recent LLVM version (and/or with a recent GCC version) - so we can see if any LCOV issues are present, or if all the weird artifacts you see are due to ancient LCOV versions, compiler/toolchain issues that are addressed in newer releases, or are bugs in either the toolchain, LCOV, or both.

I hope this helps

Henry

PS: still limited access/limited time.

[henry2cox]

If you are still seeing a report which you think is not correct, please append a testcase which exhibits the issue, or describe the problem in quite significant detail.
Note that lcov is simply displaying the information generated by llvm/llvm-cov - so, if llvm claims that a non-code line is or is not covered, then lcov will display that (however, see the --filter option in the man page).

If there is no longer a problem: please go ahead and close this issue.
Thanks
Henry

[henry2cox]

No updates or clarification for a month - so I assume the issue is resolved.
Closing now.

If you believe there is still a problem: please either reopen this issue or file a new one.
Please include a testcase which illustrates the problem.
Thanks
Henry