Logging into kafka topic via log4j2

[msviba]

Hi,
I'm trying many hours to use log4j2 with Kafka appender and json layout. It works fine only in quarkus:dev. But it doesn't when uber-jar is assembled. There is a conflict in Log4j2Plugins.dat. Assembled (quarkus:build) uber-jar has a version of Log4j2Plugins.dat from jsonLayout library. But I need probably the Log4j2Plugins.dat from log4j-core.

I've tried:

• to exclude Log4j2Plugins.dat from jar via maven quarkus plugin but I can't chose which one
• to use 'native' Quarkus ELK logging, but I needs another container (more complex ci/cd, more memory&cpu)

I have NOT tried to assemble my own library with working combination of dependencies (again more complex).

Has someone please any advice? Thanks a lot :)

relevant piece of pom.xml:

  <properties>
    <surefire-plugin.version>3.1.2</surefire-plugin.version>
    <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
    <quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
    <quarkus.platform.version>3.6.6</quarkus.platform.version>
    <maven.compiler.source>21</maven.compiler.source>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.target>21</maven.compiler.target>
    <log4j.version>2.20.0</log4j.version>
    <ojdbc8.version>21.11.0.0</ojdbc8.version>
  </properties>
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>${quarkus.platform.group-id}</groupId>
        <artifactId>${quarkus.platform.artifact-id}</artifactId>
        <version>${quarkus.platform.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.apache.kafka</groupId>
      <artifactId>kafka-log4j-appender</artifactId>
      <version>3.6.1</version>
      <exclusions>
        <exclusion>
          <groupId>org.slf4j</groupId>
          <artifactId>slf4j-api</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.slf4j</groupId>
          <artifactId>slf4j-reload4j</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-layout-template-json</artifactId>
      <version>${log4j.version}</version>
    </dependency>

    <dependency>
      <groupId>net.logstash.log4j</groupId>
      <artifactId>jsonevent-layout</artifactId>
      <version>1.7</version>
    </dependency>

    <dependency>
      <groupId>org.projectlombok</groupId>
      <artifactId>lombok</artifactId>
      <version>1.18.30</version>
    </dependency>

[quarkus-bot[bot]]

/cc @alesj (kafka), @cescoffier (kafka), @ozangunalp (kafka)

[msviba]

Workaround:
Create a new project with pom.xml where will be log4j2 related artifacts what you need. Use maven shade plugin https://github.com/edwgiz/maven-shaded-log4j-transformer. Install assembled artifact to your repository.
Than use just this newly assembled artifact instead of set of log4j dependencies.