Found 0 tests in Kotlin project

[xgouchet]

I'm trying to run PITest with the JUnit 5 plugin in an Android/Kotlin project.
I'm applying the JUnit 5 project using the recommended approach

junit5PluginVersion = '0.12'

But when I run PITest, the following outputs is given. It does find my 887 test classes, and can generate mutation, but somehow it doesn't recognizes the tests from my Kotlin classes. Is there a way to make the plugin more verbose as to whay it doesn't find my @Test annotated methods ?

3:48:27 PM PIT >> INFO : MINION : 3:48:27 PM PIT >> FINE : Expecting 887 tests classes from parent
3:48:27 PM PIT >> INFO : MINION : 3:48:27 PM PIT >> FINE : Tests classes received
3:48:27 PM PIT >> INFO : MINION : 3:48:27 PM PIT >> INFO : Checking environment
3:48:29 PM PIT >> INFO : MINION : 3:48:29 PM PIT >> INFO : Found  0 tests
3:48:29 PM PIT >> INFO : MINION : 3:48:29 PM PIT >> INFO : Dependency analysis reduced number of potential tests by 0
3:48:29 PM PIT >> INFO : MINION : 3:48:29 PM PIT >> INFO : 0 tests received
3:48:29 PM PIT >> INFO : MINION : 3:48:29 PM PIT >> FINE : Running 0 units

[szpak]

As mentioned in README of gradle-pitest-plugin:

Can I use gradle-pitest-plugin with my Android application?

you should use the gradle-pitest-plugin fork intended for use in the Android projects. You might want to refer to the configuration examples in that project.

[xgouchet]

Hi @szpak ,

I am using the Android fork, which is a wrapper that simply finds the source and test classes differently. That fork works correctly with JUnit 4 tests, but applying this plugin doesn't work.

[szpak]

Ok, it was misled by the link to the original project webpage.

I don't know if the fork support JUnit 5 tests with Android. With enabling verbose mode in PIT (and --info in Gradle) you should see a message that JUnit 5 plugin is being enabled. Do you have one? If not you might paste PIT related output here.

@koral-- do you have a functional test with JUnit 5 that could confirm that at least in the basic case works?

[szpak]

Btw, @xgouchet here you have a sample JUnit 5 and Kotlin project which works. It's not an Android project, but maybe you will spot any other difference.

[xgouchet]

So there is an issue in the Android version, which doesn't handle the junit5PluginVersion attribute, but even when specifying manually the testPlugin and adding the Junit5 artifact in the buildScript dependencies, it doesn't seem to work.
What line should I be seeing in the logs that confirms that the JUnit5 plugin is indeed loaded and used ?

In the log I see this line :

Running report with ReportOptions
    [
        targetClasses=[com.example.*],
        excludedMethods=[],
        excludedClasses=[],
        excludedTestClasses=[],
        codePaths=[
            ~/…/build/intermediates/javac/debug/classes,
            ~/…/build/tmp/kotlin-classes/debug
        ],
        reportDir=~/…/build/reports/pitest,
        historyInputLocation=null,
        historyOutputLocation=null,
        sourceDirs=[
            ~/…/src/main/java,
            ~/…/src/main/kotlin
        ],
        classPathElements=[
            ~/.gradle/caches/modules-2/files-2.1/org.pitest/pitest-junit5-plugin/0.12/6e1ee587a3457b0e7d636a5ce2e554fac62ec99/pitest-junit5-plugin-0.12.jar,
            ~/.gradle/caches/modules-2/files-2.1/org.pitest/pitest/1.5.0/f8e6d4738f0fca684aaec7c8e0e78e84670fa1bc/pitest-1.5.0.jar,
            ~/…/build/pitest-android-29-default-values.jar,
            ~/…/build/intermediates/sourceFolderJavaResources/debug,
            ~/…/build/intermediates/sourceFolderJavaResources/test/debug,
            ~/…/build/intermediates/java_res/debug/out,
            ~/…/build/intermediates/java_res/debugUnitTest/out,
            ~/…/build/intermediates/unitTestConfig/test/debug,
            ~/…/build/intermediates/compile_and_runtime_not_namespaced_r_class_jar/debugUnitTest/R.jar,
            ~/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-launcher/1.5.2/bca9d635ae55b37540e6e26e3e08ea0977516fe3/junit-platform-launcher-1.5.2.jar,
            ~/.gradle/caches/modules-2/files-2.1/org.junit.vintage/junit-vintage-engine/5.5.2/c3422d2aed908952f9db3e7f3a396776e5a8e017/junit-vintage-engine-5.5.2.jar,
            ~/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-engine/1.5.2/d5697f6ebe0b4d08c0210b5b98b4e1a40f40dfc6/junit-platform-engine-1.5.2.jar,
            ~/.gradle/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter/5.5.2/f8561a498ec26b24a3a64aebb6b2307fed000a33/junit-jupiter-5.5.2.jar,
            ~/.gradle/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter-params/5.5.2/e0659722923bd9fdfa08602e2da22bd5a9d354e8/junit-jupiter-params-5.5.2.jar,
            ~/.gradle/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter-api/5.5.2/6393db7e4c0265152d8fc4ff146633d1a7d36c47/junit-jupiter-api-5.5.2.jar,
            ~/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-commons/1.5.2/9703df63b65d415b2a027d43ce908c625b3aedce/junit-platform-commons-1.5.2.jar,
            ~/…/build/intermediates/javac/debug/classes,
            ~/…/build/tmp/kotlin-classes/debug,
            ~/…/build/tmp/kapt3/classes/debug,
            ~/…/build/tmp/kotlin-classes/debugUnitTest,
            ~/…/build/tmp/kapt3/classes/debugUnitTest,
            ~/…/build/intermediates/unmocked-androiddd-sdk-android.jar,
            ~/…/build/intermediates/javac/debugUnitTest/classes,
            ~/…/build/intermediates/compile_only_not_namespaced_r_class_jar/debug/R.jar
        ],
        mutators=[STRONGER],
        features=[],
        dependencyAnalysisMaxDistance=-1,
        jvmArgs=[-Djava.awt.headless=true],
        numberOfThreads=8,
        timeoutFactor=1.25,
        timeoutConstant=4000,
        targetTests=[^com\.example\..*Test$],
        loggingClasses=[kotlin.jvm.internal],
        maxMutationsPerClass=0,
        verbose=true,
        failWhenNoMutations=true,
        outputs=[XML, HTML],
        groupConfig=TestGroupConfig [excludedGroups=[],
        includedGroups=[]],
        fullMutationMatrix=false,
        mutationUnitSize=0,
        shouldCreateTimestampedReports=false,
        detectInlinedCode=false,
        exportLineCoverage=false,
        mutationThreshold=0,
        coverageThreshold=0,
        mutationEngine=gregor,
        javaExecutable=null,
        includeLaunchClasspath=false,
        properties={},
        maxSurvivors=-1,
        excludedRunners=[],
        includedTestMethods=[],
        testPlugin=junit5,
        useClasspathJar=false,
        skipFailingTests=false
    ]

Is there something missing ?

[szpak]

It in general looks ok. There could be some Android-related quirks. Maybe @koral-- will have any idea.

[koral--]

I confirm that it junit5 plugin does not work with Android projects.
I'll try to figure out what is the exact reason.

[koral--]

Quite a long time but I've managed to find a time to investigate that.

It seems that the workaround consists of 2 steps:

• use old-fashioned way to apply junit5 pitest plugin (pitest.testPlugin = "junit5" and pitest "org.pitest:pitest-junit5-plugin:0.12" buildscript dependency)
• add junit5 runtime to pitestTestCompile buildscript dependency

See the commit autolinked above and the whole functional test where it is located.

[szpak]

Nice @koral--!

Based on your research, why pitestTestCompile "org.junit.jupiter:junit-jupiter-engine:$junit5Version" is required? Is it a problem with junit5 detection by the plugin (only in Android projects?) or that dependency should be added by default for some other reasons?

[koral--]

Normally (not taking pitest into account) that dependency has to be added to testRuntimeOnly configuration and there is a 3rd party gradle plugin to make it work in Android projects: https://github.com/mannodermaus/android-junit5#setup.

It seems that the more elegant solution in the plugin (apart from making junit5PluginVersion work) is to add everything from test runtime classpath to additionalClasspath of pitest task.

[koral--]

I was able to fix that issue.
Now it is possible to just specify junit5PluginVersion without any additional properties specific to pitest.