Allow kafka-clients 3.9.x with Spring-Kafka 3.3.x

[bgK]

The test artifacts of kafka-clients versions 3.7.0 to 3.8.1 include a junit-platform.properties file. JUnit only supports a single junit-platform.properties. As a result, the file provided by kafka-clients can override the application provided file depending on classpath order. This has been fixed in 3.9.0 with KAFKA-17121.

In our application the junit-platform.properties file enables an extention for creating a test database.
When upgrading from Spring-Kafka 3.1.x to 3.3.x, kafka-clients was updated and the extension was no longer executed.
As a workaround, we downgraded kafka-clients to 3.6.2. This seems to work even though it's not documented as supported.

It would be nice if Spring-Kafka 3.3.x could support kafka-clients version 3.9.0 as a proper fix.

[sobychacko]

@bgK Did you try upgrading your application to use 3.9.0 kafka client? We ran into some other issues with 3.9.0, that we will fix, but generally speaking you should be able to update to 3.9.0 on the application side.

[bgK]

@sobychacko I'm getting the following error with kafka-clients 3.9.0 when starting the embedded Kafka broker:

java.lang.NoSuchMethodError: 'kafka.testkit.KafkaClusterTestKit$Builder kafka.testkit.KafkaClusterTestKit$Builder.setConfigProp(java.lang.String, java.lang.String)'

	at org.springframework.kafka.test.EmbeddedKafkaKraftBroker.lambda$start$0(EmbeddedKafkaKraftBroker.java:219)
	at java.base/java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1605)
	at java.base/java.util.Properties.forEach(Properties.java:1467)
	at org.springframework.kafka.test.EmbeddedKafkaKraftBroker.start(EmbeddedKafkaKraftBroker.java:219)
	at org.springframework.kafka.test.EmbeddedKafkaKraftBroker.afterPropertiesSet(EmbeddedKafkaKraftBroker.java:204)
	at org.springframework.kafka.test.context.EmbeddedKafkaContextCustomizer.customizeContext(EmbeddedKafkaContextCustomizer.java:135)
	at org.springframework.test.context.support.AbstractContextLoader.customizeContext(AbstractContextLoader.java:189)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:218)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:110)
	at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:212)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:225)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:152)
	at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:130)
	at org.springframework.test.context.bean.override.BeanOverrideTestExecutionListener.injectFields(BeanOverrideTestExecutionListener.java:87)
	at org.springframework.test.context.bean.override.BeanOverrideTestExecutionListener.prepareTestInstance(BeanOverrideTestExecutionListener.java:58)
	at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:260)
	at org.springframework.test.context.junit.jupiter.SpringExtension.postProcessTestInstance(SpringExtension.java:160)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:194)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:611)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)

The method signature changed from setConfigProp(String, String) to setConfigProp(String, Object).

[sobychacko]

Thanks for that report. We will try to address it soon.