Log4j incompatible with recent kubernetes-client

[Amya28]

Description

While using the latest log4j release version ; we encountered the following error :

ERROR StatusConsoleListener Unable to create Lookup for k8s
java.lang.NoSuchMethodError: 'io.fabric8.kubernetes.client.ConfigFluent io.fabric8.kubernetes.client.ConfigBuilder.withRollingTimeout(long)'

Debugging we found that in latest version of kubernetes-client-api they removed the getRollingTimeout calls which is still referred in latest version of log4j-Kubernetes jar.

fabric8io/kubernetes-client@f12e14b

Even tough the Release notes https://github.com/apache/logging-log4j2/releases specify the Kubernetes client to version 5.12.4

Can anyone have a look at it. we need to specify the version explicitly for kubernetes client under log4j pom.xmls.

[A clear and concise description of what the bug is.]

Configuration

Version: [[2.22.0] (https://github.com/apache/logging-log4j2/releases/tag/rel%2F2.22.0)]

Operating system: [Windows 10]

JDK: [17]

Logs

[Stacktraces, errors, etc. relevant applications logs.]

ERROR StatusConsoleListener Unable to create Lookup for k8s
 java.lang.NoSuchMethodError: 'io.fabric8.kubernetes.client.ConfigFluent io.fabric8.kubernetes.client.ConfigBuilder.withRollingTimeout(long)'
        at org.apache.logging.log4j.kubernetes.KubernetesClientBuilder.kubernetesClientConfig(KubernetesClientBuilder.java:69)
        at org.apache.logging.log4j.kubernetes.KubernetesClientBuilder.createClient(KubernetesClientBuilder.java:35)
        at org.apache.logging.log4j.kubernetes.KubernetesLookup.initialize(KubernetesLookup.java:89)
        at org.apache.logging.log4j.kubernetes.KubernetesLookup.<init>(KubernetesLookup.java:70)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at org.apache.logging.log4j.core.util.ReflectionUtil.instantiate(ReflectionUtil.java:189)
        at org.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:86)
        at org.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:105)
        at org.apache.logging.log4j.core.config.AbstractConfiguration.<init>(AbstractConfiguration.java:135)
        at org.apache.logging.log4j.core.config.DefaultConfiguration.<init>(DefaultConfiguration.java:46)
        at org.apache.logging.log4j.core.LoggerContext.<init>(LoggerContext.java:84)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.createContext(ClassLoaderContextSelector.java:254)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.locateContext(ClassLoaderContextSelector.java:218)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:136)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:123)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:117)
        at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:150)
        at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
        at org.apache.logging.log4j.LogManager.getContext(LogManager.java:196)
        at org.apache.commons.logging.LogAdapter$Log4jLog.<clinit>(LogAdapter.java:146)
        at org.apache.commons.logging.LogAdapter$Log4jAdapter.createLog(LogAdapter.java:113)
        at org.apache.commons.logging.LogAdapter.createLog(LogAdapter.java:95)
        at org.apache.commons.logging.LogFactoryService.getInstance(LogFactoryService.java:52)
        at org.apache.commons.logging.LogFactoryService.getInstance(LogFactoryService.java:47)
        at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:655)


## Reproduction

[An isolated test reproducing the test.
JUnit tests similar to the ones in the code base are extremely appreciated.]

[Amya28]

https://github.com/apache/logging-log4j2/blob/rel/2.22.1/log4j-kubernetes/pom.xml

This doesnot specify the kubernetes-client version hence, it is fetching to the latest version of kubernetes-client : 6.9.2

[ppkarwasz]

@Amya28,

thank you for the issue report. Currently log4j-kubernetes is compiled against version 5.12.4:

logging-log4j2/log4j-parent/pom.xml

Line 124 in 8469975

<kubernetes-client.version>5.12.4</kubernetes-client.version>

but it compiles fine up to version 6.1.1 of the Kubernetes libraries.

Kubernetes seems to have performed several breaking API changes in minor releases without waiting for a major release:

• in version 6.2.0 the method io.fabric8.kubernetes.api.model.ObjectMeta#getClusterName disappeared,
• as you noticed in version 6.5.0 the methods io.fabric8.kubernetes.client.Config#getRollingTimeout and io.fabric8.kubernetes.client.ConfigBuilder#withRollingTimeout disappeared.

I have reported the problem upstream.

[Amya28]

@Amya28,

thank you for the issue report. Currently log4j-kubernetes is compiled against version 5.12.4:

logging-log4j2/log4j-parent/pom.xml

Line 124 in 8469975

<kubernetes-client.version>5.12.4</kubernetes-client.version>

but it compiles fine up to version 6.1.1 of the Kubernetes libraries.

Kubernetes seems to have performed several breaking API changes in minor releases without waiting for a major release:

• in version 6.2.0 the method io.fabric8.kubernetes.api.model.ObjectMeta#getClusterName disappeared,
• as you noticed in version 6.5.0 the methods io.fabric8.kubernetes.client.Config#getRollingTimeout and io.fabric8.kubernetes.client.ConfigBuilder#withRollingTimeout disappeared.

I have reported the problem upstream.

Thanks @ppkarwasz for the prompt reply. Yes, Kubernetes have performed several breaking API changes in their latest release.

But it does not seem that log4j-kubernetes is compiled against version 5.12.4
The pom.xml of log4j-kubernetes does not specify the version tag and hence it is fetching the latest 'kubernetes-client' version.

Dependency tree:

[INFO] +- org.apache.logging.log4j:log4j-kubernetes:jar:2.21.1:compile
[INFO] |  \- io.fabric8:kubernetes-client:jar:6.9.2:compile
[INFO] |     +- io.fabric8:kubernetes-client-api:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-core:jar:6.9.2:compile
[INFO] |     |  |  \- io.fabric8:kubernetes-model-common:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-gatewayapi:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-resource:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-rbac:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-admissionregistration:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-apps:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-autoscaling:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-apiextensions:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-batch:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-certificates:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-coordination:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-discovery:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-events:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-extensions:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-flowcontrol:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-networking:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-metrics:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-policy:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-scheduling:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-storageclass:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-node:jar:6.9.2:compile

We need to use the<kubernetes-client.version>5.12.4</kubernetes-client.version> throughout.

[ppkarwasz]

@Amya28,

In Maven you need to specify an artifact's version, it throws an error if you don't. The reason you don't see it in log4j-kubernetes/pom.xml is that we define it in log4j-parent/pom.xml dependency management section.

Your project can of course override that version, which is probably what happens.

[vy]

@Amya28, are there any particular reasons you choose KubernetesLookup over exporting associated Kubernetes configuration to environment variables and accessing them using EnvironmentLookup, e.g., ${env:KUBERNETES_CLUSTER_NAME}?

I am curious about your use case for following reasons:

• We considered dropping KubernetesLookup in Log4j 3.0.0
• Using EnvironmentLookup helps you avoid any kubernetes-client dependencies
• Judging from my personal experience, injecting K8s config into apps through env vars is a common industry practice

Hence, I want to understand your rationale for deviating from this.

[ppkarwasz]

@vy,

After the explanations provided by @manusa in fabric8io/kubernetes-client#5682, I think that the best solution would be to either move log4j-kubernetes to a Fabric8 repo or to a repo, where both our teams have access.

The lifecycle of log4j-kubernetes should clearly follow the lifecycle of kubernetes-client instead of log4j-core.

[vy]

@ppkarwasz, yes, I have seen fabric8io/kubernetes-client#5682 discussion. Thanks for reporting the issue to the upstream. I agree that if kubernetes-client wants to adopt log4j-kubernetes, they should.

Yet I still want to understand the use case of @Amya28. Because, maybe, there should not be a KubernetesLookup at all? I don't feel comfortable with moving the maintenance responsibility of a component that should not have existed in the first place on to somebody else's shoulders.

[Amya28]

@ppkarwasz @vy
There's no KubernetesLookup.

In our case, we are working with spring boot and through Spring cloud the kubernetes version is set to latest 6.9.2 by framework which is causing above mentioned issue.

[ppkarwasz]

@Amya28,

If you don't use the KubernetesLookup (i.e. you don't have ${k8s:...} placeholders in your config file), you can remove the log4j-kubernetes dependency, since it does not provide anything else.

[ppkarwasz]

I am closing this, since a new kubernetes-log4j lookup is available since version 6.11.0 of Kubernetes Client (cf. announcement).