Skip to content

Group logs by request - GCP #50319

@GuidoFarport

Description

@GuidoFarport

Describe the bug

I have a problem. I switched to using Quarkus for my applications (from Spring Boot), but I'm having trouble because I can't figure out why everything I had configured regarding the appearance of the logs isn't working.

Let me try to explain better. In my Spring projects, I had configured Logback + Spring Cloud Logging Lib + Spring Cloud Trace Lib to allow me to configure, based on the active Spring profile, how to append logs once deployed on Google Cloud Run.

I have seen that Quarkus has attempted to standardise the use of logs using Jboss Log Manager, and that would be fine with me, but I would like my logs to be grouped under my incoming request.

Example:

Image

As you can see, all my logs are not grouped under the input request, i've a trace under te request but not under all other logs.

This is my example controller:

Image

Log lib:

Image

To do this with my Google project, I saw that what Google needed to help group everything together was the combination of logback + the spring-google-cloud-trace library, spring-google-cloud-logging, which allowed me to do two things:

  1. add a trace/trace ID to each log
  2. hang everything on Google Cloud.

Expected behavior

What I expect is that there is a way to allow me to add the trace to the request log and to all logs within this request...

Image

Like you see all log under the request have the trace attached..

But with my quarkus app i can't replicate this..

Actual behavior

No response

How to Reproduce?

To reproduce this you need to deploy the solution under google cloud project.

I can't attach here all my solution to do this.

But i can paste here all info you need..

For example:

the pom is:

<?xml version="1.0" encoding="UTF-8"?>
<project>
    <modelVersion>4.0.0</modelVersion>

    <groupId>it.exante</groupId>
    <artifactId>exapp-api-core</artifactId>
    <version>0.1.1</version>

    <properties>
        <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
        <quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
        <quarkus.platform.version>3.26.1</quarkus.platform.version>
        <lombok.version>1.18.34</lombok.version>
        <common.math.version>2.2</common.math.version>
        <compiler-plugin.version>3.11.0</compiler-plugin.version>
        <surefire-plugin.version>3.1.2</surefire-plugin.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>21</maven.compiler.source>
        <maven.compiler.target>21</maven.compiler.target>
        <maven.compiler.parameters>true</maven.compiler.parameters>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>${quarkus.platform.artifact-id}</artifactId>
                <version>${quarkus.platform.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>quarkus-google-cloud-services-bom</artifactId>
                <version>${quarkus.platform.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-rest-jackson</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-jdbc-mariadb</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-spring-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-spring-web</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-undertow</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-spring-di</artifactId>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>${lombok.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-logging-json</artifactId>
        </dependency>
<!--        <dependency>-->
<!--            <groupId>io.quarkiverse.logging.logback</groupId>-->
<!--            <artifactId>quarkus-logging-logback</artifactId>-->
<!--        </dependency>-->
<!--        <dependency>-->
<!--            <groupId>app.fp8.logging</groupId>-->
<!--            <artifactId>gcloud-json-logging</artifactId>-->
<!--            <version>1.0-SNAPSHOT</version>-->
<!--        </dependency>-->
        <!-- https://mvnrepository.com/artifact/com.google.cloud/google-cloud-trace -->
        <dependency>
            <groupId>com.google.cloud</groupId>
            <artifactId>google-cloud-trace</artifactId>
            <version>2.74.0</version>
        </dependency>



        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-math</artifactId>
            <version>${common.math.version}</version>
        </dependency>

        <!--TEST LIBs-->
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-container-image-podman</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-junit5</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.rest-assured</groupId>
            <artifactId>rest-assured</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-junit5-mockito</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${compiler-plugin.version}</version>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${surefire-plugin.version}</version>
                <configuration>
                    <systemPropertyVariables>
                        <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                        <maven.home>${maven.home}</maven.home>
                    </systemPropertyVariables>
                </configuration>
            </plugin>
            <plugin>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>quarkus-maven-plugin</artifactId>
                <version>${quarkus.platform.version}</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>build</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>versions-maven-plugin</artifactId>
                <version>2.12.0</version>
                <configuration>
                    <allowSnapshots>true</allowSnapshots>
                    <includes>
                        <include>it.exante:*</include>
                    </includes>
                </configuration>
            </plugin>
        </plugins>
        <extensions>
            <extension>
                <groupId>com.google.cloud.artifactregistry</groupId>
                <artifactId>artifactregistry-maven-wagon</artifactId>
                <version>2.1.1</version>
            </extension>
        </extensions>
    </build>


    <profiles>
        <profile>
            <id>native</id>
            <activation>
                <property>
                    <name>native</name>
                </property>
            </activation>
            <properties>
                <quarkus.native.enabled>true</quarkus.native.enabled>
                <quarkus.package.jar.enabled>false</quarkus.package.jar.enabled>
            </properties>
            <build>
                <plugins>
                    <plugin>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>${surefire-plugin.version}</version>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>integration-test</goal>
                                    <goal>verify</goal>
                                </goals>
                                <configuration>
                                    <systemPropertyVariables>
                                        <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path>
                                        <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                                        <maven.home>${maven.home}</maven.home>
                                    </systemPropertyVariables>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>
</project>


Dockerfile


####
# This Dockerfile is used in order to build a container that runs the Quarkus application in JVM mode
#
# Before building the container image run:
#
# ./mvnw package
#
# Then, build the image with:
#
# docker build -f src/main/docker/Dockerfile.jvm -t quarkus/getting-started-jvm .
#
# Then run the container using:
#
# docker run -i --rm -p 8080:8080 quarkus/getting-started-jvm
#
# If you want to include the debug port into your docker image
# you will have to expose the debug port (default 5005 being the default) like this :  EXPOSE 8080 5005.
# Additionally you will have to set -e JAVA_DEBUG=true and -e JAVA_DEBUG_PORT=*:5005
# when running the container
#
# Then run the container using :
#
# docker run -i --rm -p 8080:8080 quarkus/getting-started-jvm
#
# This image uses the `run-java.sh` script to run the application.
# This scripts computes the command line to execute your Java application, and
# includes memory/GC tuning.
# You can configure the behavior using the following environment properties:
# - JAVA_OPTS: JVM options passed to the `java` command (example: "-verbose:class") - Be aware that this will override
# the default JVM options, use `JAVA_OPTS_APPEND` to append options
# - JAVA_OPTS_APPEND: User specified Java options to be appended to generated options
#   in JAVA_OPTS (example: "-Dsome.property=foo")
# - JAVA_MAX_MEM_RATIO: Is used when no `-Xmx` option is given in JAVA_OPTS. This is
#   used to calculate a default maximal heap memory based on a containers restriction.
#   If used in a container without any memory constraints for the container then this
#   option has no effect. If there is a memory constraint then `-Xmx` is set to a ratio
#   of the container available memory as set here. The default is `50` which means 50%
#   of the available memory is used as an upper boundary. You can skip this mechanism by
#   setting this value to `0` in which case no `-Xmx` option is added.
# - JAVA_INITIAL_MEM_RATIO: Is used when no `-Xms` option is given in JAVA_OPTS. This
#   is used to calculate a default initial heap memory based on the maximum heap memory.
#   If used in a container without any memory constraints for the container then this
#   option has no effect. If there is a memory constraint then `-Xms` is set to a ratio
#   of the `-Xmx` memory as set here. The default is `25` which means 25% of the `-Xmx`
#   is used as the initial heap size. You can skip this mechanism by setting this value
#   to `0` in which case no `-Xms` option is added (example: "25")
# - JAVA_MAX_INITIAL_MEM: Is used when no `-Xms` option is given in JAVA_OPTS.
#   This is used to calculate the maximum value of the initial heap memory. If used in
#   a container without any memory constraints for the container then this option has
#   no effect. If there is a memory constraint then `-Xms` is limited to the value set
#   here. The default is 4096MB which means the calculated value of `-Xms` never will
#   be greater than 4096MB. The value of this variable is expressed in MB (example: "4096")
# - JAVA_DIAGNOSTICS: Set this to get some diagnostics information to standard output
#   when things are happening. This option, if set to true, will set
#  `-XX:+UnlockDiagnosticVMOptions`. Disabled by default (example: "true").
# - JAVA_DEBUG: If set remote debugging will be switched on. Disabled by default (example:
#    true").
# - JAVA_DEBUG_PORT: Port used for remote debugging. Defaults to 5005 (example: "8787").
# - CONTAINER_CORE_LIMIT: A calculated core limit as described in
#   https://www.kernel.org/doc/Documentation/scheduler/sched-bwc.txt. (example: "2")
# - CONTAINER_MAX_MEMORY: Memory limit given to the container (example: "1024").
# - GC_MIN_HEAP_FREE_RATIO: Minimum percentage of heap free after GC to avoid expansion.
#   (example: "20")
# - GC_MAX_HEAP_FREE_RATIO: Maximum percentage of heap free after GC to avoid shrinking.
#   (example: "40")
# - GC_TIME_RATIO: Specifies the ratio of the time spent outside the garbage collection.
#   (example: "4")
# - GC_ADAPTIVE_SIZE_POLICY_WEIGHT: The weighting given to the current GC time versus
#   previous GC times. (example: "90")
# - GC_METASPACE_SIZE: The initial metaspace size. (example: "20")
# - GC_MAX_METASPACE_SIZE: The maximum metaspace size. (example: "100")
# - GC_CONTAINER_OPTIONS: Specify Java GC to use. The value of this variable should
#   contain the necessary JRE command-line options to specify the required GC, which
#   will override the default of `-XX:+UseParallelGC` (example: -XX:+UseG1GC).
# - HTTPS_PROXY: The location of the https proxy. (example: "[email protected]:8080")
# - HTTP_PROXY: The location of the http proxy. (example: "[email protected]:8080")
# - NO_PROXY: A comma separated lists of hosts, IP addresses or domains that can be
#   accessed directly. (example: "foo.example.com,bar.example.com")
#
###
FROM registry.access.redhat.com/ubi9/openjdk-21:1.23

ENV LANGUAGE='en_US:en'


# We make four distinct layers so if there are application changes the library layers can be re-used
COPY --chown=185 target/quarkus-app/lib/ /deployments/lib/
COPY --chown=185 target/quarkus-app/*.jar /deployments/
COPY --chown=185 target/quarkus-app/app/ /deployments/app/
COPY --chown=185 target/quarkus-app/quarkus/ /deployments/quarkus/

EXPOSE 8080
USER 185
ENV JAVA_OPTS="-Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
ENV JAVA_APP_JAR="/deployments/quarkus-run.jar"

ENTRYPOINT [ "/opt/jboss/container/java/run/run-java.sh" ]


my properties:

# Quarkus Configuration file
quarkus.datasource.db-kind=mariadb
quarkus.datasource.username=${DB_USER:test}
quarkus.datasource.password=${DB_PSW:test}
quarkus.datasource.jdbc.url=jdbc:mariadb://${DB_HOST:localhost}:3306/${DB_NAME:test}
quarkus.datasource.db-version=5.7.13
quarkus.hibernate-orm.schema-management.strategy=none

my properties.dev:

#quarkus.http.port=8081
quarkus.log.level=INFO
quarkus.log.category."it.exante".level=DEBUG
quarkus.log.console.json.enabled=true

quarkus.hibernate-orm.validate-in-dev-mode=false


#quarkus.datasource.devservices.show-logs=true
quarkus.hibernate-orm.log.sql=true
quarkus.devservices.enabled=false

Let me know if you need some other info.

Output of uname -a or ver

No response

Output of java -version

21

Quarkus version or git rev

3.26.1

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

Metadata

Metadata

Assignees

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions