Adding JaCoCo support for integration tests

[johnmanko]

I'm looking to add JaCoCo to my integration tests, but Quarkus is failing to create/initialize a Redis service.

Error: "this.keyCommands" is null

I'm basically following the guides as close as I can to get this working. Here is the Redis service I'm using:

import io.quarkus.redis.datasource.RedisDataSource;
import io.quarkus.redis.datasource.keys.KeyCommands;
import io.quarkus.redis.datasource.value.ValueCommands;
import jakarta.enterprise.context.ApplicationScoped;
import java.util.Collection;
import java.util.Collections;
import java.util.Optional;
import jakarta.inject.Inject;
import org.jboss.logging.Logger;

@ApplicationScoped
public class RedisService {

    @Inject
    Logger log;

    KeyCommands<String> keyCommands;
    ValueCommands<String, ObjectLockRecord> valueCommands;

    public RedisService(RedisDataSource ds) {

        keyCommands = ds.key();
        valueCommands = ds.value(DataRecord.class);

    }

    public Collection<String> getDataKeys() throws RedisException {

        try {
            return this.keyCommands.keys("*");
        } catch (Exception ex) {
            log.error("Error obtaining data keys", ex);
            throw new RedisException(ex);
        }

    }

}

Maven configuration:

    <dependencies>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-rest</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-arc</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-redis-client</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-rest-jackson</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-smallrye-openapi</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-junit5</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.rest-assured</groupId>
            <artifactId>rest-assured</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>jakarta.servlet</groupId>
            <artifactId>jakarta.servlet-api</artifactId>
            <version>6.0.0</version>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-junit-jupiter</artifactId>
            <version>5.6.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-junit5-mockito</artifactId>
            <version>3.10.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-jacoco</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${surefire-plugin.version}</version>
                <dependencies>
                    <dependency>
                        <groupId>me.fabriciorby</groupId>
                        <artifactId>maven-surefire-junit5-tree-reporter</artifactId>
                        <version>0.1.0</version>
                    </dependency>
                </dependencies>
                <configuration>
                    <reportFormat>plain</reportFormat>
                    <systemPropertyVariables>
                        <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                        <maven.home>${maven.home}</maven.home>
                    </systemPropertyVariables>
                    <statelessTestsetReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
                        <disable>false</disable>
                        <version>3.0</version>
                        <usePhrasedFileName>false</usePhrasedFileName>
                        <usePhrasedTestSuiteClassName>true</usePhrasedTestSuiteClassName>
                        <usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
                        <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
                    </statelessTestsetReporter>
                    <consoleOutputReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5ConsoleOutputReporter">
                        <disable>true</disable>
                    </consoleOutputReporter>
                    <statelessTestsetInfoReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter">
                        <disable>false</disable>
                        <usePhrasedFileName>false</usePhrasedFileName>
                        <usePhrasedClassNameInRunning>true</usePhrasedClassNameInRunning>
                        <usePhrasedClassNameInTestCaseSummary>true</usePhrasedClassNameInTestCaseSummary>
                    </statelessTestsetInfoReporter>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>${jacoco.version}</version>
                <executions>
                    <execution>
                        <id>default-prepare-agent-integration</id>
                        <goals>
                            <goal>prepare-agent-integration</goal>
                        </goals>
                        <configuration>
                            <destFile>${project.build.directory}/jacoco-quarkus.exec</destFile>
                            <append>true</append>
                        </configuration>
                    </execution>
                    <execution>
                        <id>report</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>report</goal>
                        </goals>
                        <configuration>
                            <dataFile>${project.build.directory}/jacoco-quarkus.exec</dataFile>
                            <outputDirectory>${project.build.directory}/jacoco-report</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>${surefire-plugin.version}</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                        <configuration>
                            <systemPropertyVariables>
                                <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                                <maven.home>${maven.home}</maven.home>
                                <quarkus.test.arg-line>${argLine}</quarkus.test.arg-line>
                            </systemPropertyVariables>
                        </configuration>
                    </execution>
                </executions>
                <configuration>
                    <systemPropertyVariables>
                        <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path>
                        <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                        <maven.home>${maven.home}</maven.home>
                    </systemPropertyVariables>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <profiles>
        <profile>
            <id>native</id>
            <activation>
                <property>
                    <name>native</name>
                </property>
            </activation>
            <properties>
                <skipITs>false</skipITs>
                <quarkus.native.enabled>true</quarkus.native.enabled>
                <skipTests>false</skipTests>
                <quarkus.docker.network>host</quarkus.docker.network>
            </properties>
        </profile>
    </profiles>

The JaCoCo IT docs state:

To get code coverage data from integration tests, the following requirements need to be met:

• The built artifact is a jar (and not a container or native binary).
• JaCoCo needs to be configured in your build tool.
• The application must have been built with quarkus.package.write-transformed-bytecode-to-build-output set to true

and

Sharing the same value for quarkus.test.arg-line might break integration test runs that test different types of Quarkus artifacts. In such cases, the use of Maven profiles is advised.

The build process is in two parts:

First, build and skip ITs (works).

• -DskipITs
• -Dquarkus.package.write-transformed-bytecode-to-build-output=true
• Not native

build-non-native:
  stage: build
  image:
    name: quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21
    entrypoint:
      - ""    
    docker:
      user: root  
  services:
    - name: redis
      alias: redis
    - name: docker:dind
      alias: docker
      command: ["--tls=false"]
  variables:
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  script:      
    - echo $MAVEN_CONFIG
    - MAVEN_CONFIG="" ./mvnw clean compile package verify -DskipITs -Daether.connector.https.securityMode=insecure  -Dquarkus.package.write-transformed-bytecode-to-build-output=true -s .m2/settings.xml --errors --update-snapshots -Dquarkus.redis.hosts="redis://redis"
  after_script:
    - cat ./target/quarkus.log  
  tags:
    - quarkus
  artifacts:
    name: artifacts-build-non-native
    expire_in: 1 days
    paths:
      - target
  cache:
    paths:
      - target
      - public
  except:
    - master
    - /^master-[0-9]+(\.[0-9]+)+/
    - tags

Second, run integration tests:

integration-tests-non-native:
  stage: test
  image:
    name: quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21
    entrypoint:
      - ""    
    docker:
      user: root  
  services:
    - name: redis
      alias: redis
    - name: docker:dind
      alias: docker
      command: ["--tls=false"]
  variables:
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  script:      
    - echo $MAVEN_CONFIG
    - MAVEN_CONFIG="" ./mvnw integration-test -Daether.connector.https.securityMode=insecure  -s .m2/settings.xml --errors --update-snapshots -Dquarkus.redis.hosts="redis://redis"
  after_script:
    - cat ./target/quarkus.log  
  tags:
    - quarkus
  artifacts:
    name: artifacts-build-non-native
    expire_in: 1 days
    paths:
      - target
  cache:
    paths:
      - target
      - public
  except:
    - master
    - /^master-[0-9]+(\.[0-9]+)+/
    - tags

Here is the output log:

[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.12/org.jacoco.agent-0.8.12-runtime.jar=destfile=/builds/products/services/data-service/target/jacoco-quarkus.exec,append=true
[INFO] 
[INFO] --- failsafe:3.2.5:integration-test (default) @ data-service ---
[INFO] Failsafe report directory: /builds/products/services/data-service/target/failsafe-reports
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.mydomain.service.data.dataResourceIT
Executing "/opt/mandrel/bin/java -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.12/org.jacoco.agent-0.8.12-runtime.jar=destfile=/builds/products/services/data-service/target/jacoco-quarkus.exec,append=true -Dquarkus.http.port=8081 -Dquarkus.http.ssl-port=8444 -Dtest.url=http://localhost:8081 -Dquarkus.log.file.path=/builds/products/services/data-service/target/quarkus.log -Dquarkus.log.file.enable=true -Dquarkus.log.category."io.quarkus".level=INFO -Dquarkus.redis.hosts=redis://redis -Dquarkus.test.arg-line=-javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.12/org.jacoco.agent-0.8.12-runtime.jar=destfile=/builds/products/services/data-service/target/jacoco-quarkus.exec,append=true -jar /builds/products/services/data-service/target/quarkus-app/quarkus-run.jar"
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2024-06-04 20:53:36,870 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.container-image.build" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2024-06-04 20:53:36,870 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.container-image.group" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2024-06-04 20:53:36,871 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.container-image.name" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2024-06-04 20:53:36,871 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.container-image.registry" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2024-06-04 20:53:37,719 INFO  [io.quarkus] (main) data-service 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.10.0) started in 1.207s. Listening on: http://0.0.0.0:8081
2024-06-04 20:53:37,720 INFO  [io.quarkus] (main) Profile prod activated. 
2024-06-04 20:53:37,720 INFO  [io.quarkus] (main) Installed features: [cdi, redis-client, rest, rest-jackson, smallrye-context-propagation, smallrye-openapi, vertx]
2024-06-04 20:53:38,776 ERROR [com.mydomain.ser.obj.ser.RedisService] (executor-thread-1) Error obtaining data keys: java.lang.NullPointerException: Cannot invoke "io.quarkus.redis.datasource.keys.KeyCommands.keys(String)" because "this.keyCommands" is null

I'm pretty sure I'm meeting all of the requirements.

• Step one, build the jar artifact with quarkus.package.write-transformed-bytecode-to-build-output=true
• Step two, run IT on existing built jar, providing Redis service

---

As a side note, running the Tests and ITs against a native binary works fine when not trying to add JaCoCo support to ITs:

build-native:
  stage: build
  image:
    name: quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21
    entrypoint:
      - ""    
    docker:
      user: root  
  services:
    - name: redis
      alias: redis
    - name: docker:dind
      alias: docker
      command: ["--tls=false"]
  variables:
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  script:      
    - echo $MAVEN_CONFIG
    - MAVEN_CONFIG="" ./mvnw clean compile package verify -Daether.connector.https.securityMode=insecure -s .m2/settings.xml --errors --update-snapshots -Dquarkus.redis.hosts="redis://redis" -Dnative -Dquarkus.native.container-build=false
  after_script:
    - cat ./target/quarkus.log  
  tags:
    - quarkus
  artifacts:
    name: artifacts-build-native
    expire_in: 1 days
    paths:
      - target
  cache:
    paths:
      - target
      - public
  only:
    - master
    - /^master-[0-9]+(\.[0-9]+)+/
    - tags

[quarkus-bot[bot]]

/cc @Karm (mandrel), @galderz (mandrel), @zakkak (mandrel)

[edeandrea]

We have JaCoCo integration combined with unit tests in the Quarkus Superheroes sample application.

Take a look at https://github.com/quarkusio/quarkus-super-heroes/blob/main/rest-villains/pom.xml. We also use -Dquarkus.package.write-transformed-bytecode-to-build-output=true when we want to generate the coverage info (take a look at https://github.com/quarkusio/quarkus-super-heroes/blob/main/.github/workflows/sonar-scan.yml).

[johnmanko]

I'll take a look at that this weekend and report back. Thanks.

[brianwyka]

@edeandrea , do you have a gradle example for code coverage? The documentation for gradle on this is missing:
https://quarkus.io/guides/tests-with-coverage#coverage-for-integration-tests

[edeandrea]

I'm sorry but I don't have a Gradle example.

[brianwyka]

Having all types of byte code issues, even with the -Dquarkus.package.write-transformed-bytecode-to-build-output=true option specified.

[brianwyka]

This happens when separating unit tests and integration tests into test and intTest (Quarkus creates)

[edeandrea]

I'm sorry. I thought you were talking about maven since your examples above showed a pom.xml file. Unfortunately I don't have a Gradle example.