[2.3] Test `py2lcov/py2lcov.sh` seen fail with "genhtml: ERROR: p4annotate.pm requires environment variables P4USER P4PORT P4CLIENT to be set.

[hartwork]

test.log for download here (I hope it doesn't contain anything sensitive, please notify me if it does 🙏 )

Am I doing something wrong?:

# make -j5 -j1 check 
Creating coverage files (2 tests, 5 source files)
  Source tree ......... done (950 lines, 50 functions, 3428 branches)
  Full coverage ....... done
  Target coverage ..... done
  Partial coverage .... done
  Zero coverage ....... done
Starting tests
genhtml/full.sh ................... [pass] (time 2.3s, mem 48.7MB)
genhtml/zero.sh ................... [pass] (time 2.3s, mem 48.5MB)
genhtml/demangle.sh ............... [pass] (time 5.4s, mem 40.8MB)
genhtml/relative/relative.sh ...... [pass] (time 1.1s, mem 41.1MB)
genhtml/lambda/lambda.sh .......... [pass] (time 9.3s, mem 122.8MB)
lcov/add/zero.sh .................. [pass] (time 1.6s, mem 39.4MB)
lcov/add/zero2.sh ................. [pass] (time 3.7s, mem 44.5MB)
lcov/add/full.sh .................. [pass] (time 2.6s, mem 39.4MB)
lcov/add/full2.sh ................. [pass] (time 3.4s, mem 44.7MB)
lcov/add/prune.sh ................. [pass] (time 3.8s, mem 35.8MB)
lcov/add/track.sh ................. [pass] (time 1.5s, mem 35.1MB)
lcov/misc/help.sh ................. [pass] (time 1.2s, mem 34.5MB)
lcov/misc/version.sh .............. [pass] (time 1.4s, mem 34.4MB)
lcov/summary/zero.sh .............. [pass] (time 1.8s, mem 39.3MB)
lcov/summary/full.sh .............. [pass] (time 1.7s, mem 39.4MB)
lcov/demangle/demangle.sh ......... [pass] (time 13.7s, mem 58.9MB)
lcov/exception/exception.sh ....... [pass] (time 15.3s, mem 47.9MB)
lcov/gcov-tool/path.sh ............ [pass] (time 26.7s, mem 35.7MB)
lcov/branch/branch.sh ............. [pass] (time 7.6s, mem 58.7MB)
lcov/merge/merge.sh ............... [pass] (time 11.8s, mem 34.9MB)
lcov/format/format.sh ............. [pass] (time 7.1s, mem 34.8MB)
lcov/errs/errs.sh ................. [pass] (time 30.1s, mem 35.0MB)
lcov/multiple/multiple.sh ......... [pass] (time 5.2s, mem 35.9MB)
lcov/follow/follow.sh ............. [pass] (time 6.2s, mem 35.8MB)
lcov/initializer/initializer.sh ... [pass] (time 5.2s, mem 83.2MB)
lcov/lambda/lambda.sh ............. [pass] (time 1.5s, mem 38.0MB)
py2lcov/py2lcov.sh ................ [fail] (time 3.2s, mem 40.6MB)
    Skipping 63 previous lines (see /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/tests//test.log)
    ...
    FNL:1,12,16
    FNA:0,0,main.localfunc.nested1.nested2
    FNA:1,0,main.localfunc.nested1
    FNL:0,13,14
    FNA:0,0,main.localfunc.nested1.nested2
    FNL:3,5,18
    genhtml: ERROR: (package) unable to create callback from module '/var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/scripts/p4annotate.pm': genhtml: ERROR: p4annotate.pm requires environment variables P4USER P4PORT P4CLIENT to be set.
        (use "genhtml --ignore-errors package ..." to bypass this error)
    genhtml failed
  EXITCODE ...: 1
make[2]: *** [../common.mak:116: check] Error 1
27 tests executed, 26 passed, 1 failed, 0 skipped (time 177.9s, mem 1228.5MB)
Result log stored in /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/tests/test.log
make[1]: *** [common.mak:116: check] Error 1
make: *** [Makefile:262: check] Error 2

CC #378 #380

[hartwork]

PS: Same for gendiffcov/simple/script.sh:

test.log for download here

# make -j5 -j1 check 
[..]
lcov/lambda/lambda.sh ............. [pass] (time 0.6s, mem 38.2MB)
gendiffcov/simple/script.sh ....... [fail] (time 61.0s, mem 64.2MB)
    Skipping 2376 previous lines (see /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/tests//test.log)
    ...
    genhtml --branch-coverage --parallel 0 --profile --demangle-cpp --prefix /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/tests/gendiffcov --version-script /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/scripts/getp4version --version-script --allow-missing --frame --output-directory ./annotate --annotate /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/scripts/p4annotate.pm annotate.info
    Found 4 cores.
    genhtml: ERROR: (package) unable to create callback from module '/var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/scripts/p4annotate.pm': genhtml: ERROR: p4annotate.pm requires environment variables P4USER P4PORT P4CLIENT to be set.
        (use "genhtml --ignore-errors package ..." to bypass this error)
    genhtml --branch-coverage --parallel 0 --profile --demangle-cpp --prefix /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/tests/gendiffcov --version-script /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/scripts/getp4version --version-script --allow-missing --frame --output-directory ./annotate --annotate /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/scripts/p4annotate.pm
    Found 4 cores.
    genhtml: ERROR: (package) unable to create callback from module '/var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/scripts/p4annotate.pm': genhtml: ERROR: p4annotate.pm requires environment variables P4USER P4PORT P4CLIENT to be set.
        (use "genhtml --ignore-errors package ..." to bypass this error)
    ERROR: annotate with no annotation ignore did not pass
  EXITCODE ...: 1
make[3]: *** [../../common.mak:116: check] Error 1
make[2]: *** [../common.mak:116: check] Error 1
27 tests executed, 26 passed, 1 failed, 0 skipped (time 124.7s, mem 1252.5MB)
Result log stored in /var/tmp/portage/dev-util/lcov-2.3/work/lcov-2.3/tests/test.log
make[1]: *** [common.mak:116: check] Error 1
make: *** [Makefile:262: check] Error 2

[henry2cox]

Oops :-)
Yeah - we test locally with git and perforce...
I'm not entirely sure (yet) why these pass on github and in my 'external' flow but fail for you in this instance.
I'll check if simply setting fake entries in the driver script when there are none addresses the issue.
Otherwise, could simply check and skip the tests.

For this and other issues: would be helpful to have the entire tests directory - which would include the shrapnel that might help, to debug.
It should also be possible to change directory to the failing test, then run <testname>.sh -v to enable verboisity.
I think that make VERBOSE=1 ... will do something similar - and make the test.log file more informative.

[henry2cox]

I just created PR #382 which might address the issues you ran into.
Could you try it in your environment and see if there are any fixes or new failures.
Would be good to understand why your environment seems so different.

[hartwork]

@henry2cox I have tried with the diff of pull request #382 applied now in Gentoo packaging context and I confirm that it fixes the symptoms of both #380 and #381 🎉

[hartwork]

Closing as fixed by pull request #382…