arpack-ng + coveralls KO

[fghoussen]

Could you help here: opencollab/arpack-ng#194 ?
No way to get coverage to work (lots of files filtered => biased coverage).

[JoakimSoderberg]

Well you need to be more specific and provide examples of what you mean exactly if you want me to be able to help. Which files are filtered that you don't expect to for example?

[fghoussen]

No way to reduce the problem: real life case (f77 + f90 + c + cpp)...
Look here: https://travis-ci.org/opencollab/arpack-ng/jobs/488378478 (line 574) lots of files are filtered.

Filtered files cause coverage to be not reliable (reg reported when no reg). What we expect is : do not filter files (f77 -> cpp) under "/home/travis/build/opencollab/arpack-ng/..." (all theses files should have been passed to coveralls_setup)

To reproduce the problem, you may clone at my side: https://github.com/fghoussen/arpack-ng (branch arpackmm)

If you have any clue, would be good to know...

[JoakimSoderberg]

Hey,

I have a bit hard understanding what you mean, it's easier if you list actual files and stuff. I understand it's a real world case. But I have never tried with using weird filenames like f77 and the like. I have no idea what type of code this is, and how it works with gcov and the other tools?

The output you provided is quite random. What I want is basically:

• List of files inputted
• Expected coverage
• Actual coverage output

Is it missing lines within the files? or is it skipping entire files?

I have never tried with such weird filenames so that might be the issue. The script itself is quite extensively documented with comments. Cmake isn't the best scripting language so it can be quite cumbersome at times:
https://github.com/JoakimSoderberg/coveralls-cmake/blob/master/cmake/CoverallsGenerateGcov.cmake

I would suggest adding some prints in this file and try to see where the files you talk about gets sorted out.

It was a long time I wrote this code, and I don't use coveralls myself anymore, and I don't really have time to look at this much

[fghoussen]

Could you have a quick look ?

/tmp> git clone https://github.com/fghoussen/arpack-ng -b arpackmm; cd arpack-ng
/tmp/arpack-ng> mkdir -p build; cd build; cmake -DEXAMPLES=ON -DMPI=ON -DICBEXMM=ON -DCOVERALLS=ON ..; make all test coveralls

The output you provided is quite random. What I want is basically:

List of files inputted

The whole list of files from arpack-ng (.f, .F90, .c, .cpp)

>> sed -n '684,689p' CMakeLists.txt
    # Create the coveralls target.
    # Also lists the c/cpp files for test purposes
    coveralls_setup(
        "${arpacksrc_STAT_SRCS} ${arpackutil_STAT_SRCS} ${arpacksrc_ICB} ${arpack_TST_SRC} ${arpack_EX_F_SRC} ${arpack_EX_CPP_SRC} ${parpacksrc_STAT_SRCS} ${parpackutil_STAT_SRCS} ${parpacksrc_ICB} ${parpack_TST_SRC} ${parpack_EX_SRC}" # The source files.
        ON                 # If we should upload.
        "${PROJECT_SOURCE_DIR}/cmake/") # (Optional) Alternate project cmake module path.

Expected coverage

Don't know. Never fully worked !

Actual coverage output

cmake coveralls output => looks like many files are filtered (message: "Generate JSON for non-gcov file: XXX")

Is it missing lines within the files? or is it skipping entire files?

Would say files are entirely filtered (but should not) => coverage is biased

[fghoussen]

May be related : opencollab/arpack-ng#279. If so, you may like to back this here