[KIP-974] Consider integration with Apache Kafka native images.

[k-wall]

Currently the test extension uses Quarkus based kafka-native images from https://github.com/ozangunalp/kafka-native for container based kafka broker support. Kroxylicious-junit5-ext should consider whether the work of KIP-974 https://github.com/apache/kafka/blob/trunk/docker/native/README.md should be made available, either instead of, or in addition, to the current native support.

[k-wall]

I think we can make this largely transparent to the user of the extension. If the user is using a KRaft Kafka with a version >= 3.8.0, the test framework could default to using Apache based images. Otherwise, the behaviour needs to be at today. It should be possible to override the image location in either case.

[k-wall]

I think the extension should aim to support both the JVM based Apache Kafka Images (https://cwiki.apache.org/confluence/display/KAFKA/KIP-975) and Apache Kafka Native Images. It should probably default to the latter, but if it should somehow be possible to have the test framework use the non-native images too.

[robobario]

An update on this, we want to switch to apache for 4.1 (I believe we aren't going to release any more versions of ozangunalp/kafka-native?)

SASL not working on apache native image

I found a couple of issues with the upstream native image around SASL and classes that can't be reflectively loaded.

https://issues.apache.org/jira/browse/KAFKA-19583
https://issues.apache.org/jira/browse/KAFKA-19584

Fixes for these aren't likely to go into the 4.1 release this late, so we might hope to see them in a later point release.

I think we should implement it so that we prefer apache-native for 4.0+, and if SASL features are enabled we will use the non-native image. Plus give users an annotation to influence whether we run native or not. Like:

@ExecutionVariant(Variant.NATIVE | Variant.JAVA)

With rules:

1. InVM defaults to a JAVA variant and cannot run a NATIVE variant (so in junit testcontainers should be selected)
2. Testcontainers defaults to a NATIVE variant, unless the user configures a sasl mechanism, in which case we default to JAVA
3. If the user explicitly specifies @ExecutionVariant(Variant.NATIVE) and a sasl mechanism, we fail fast.

Alternatively we could make it explicitly about containers, like, @ImageFlavour(NATIVE | JAVA) and the InVM mode wouldn't support that annotation.

apache images dont have SCRAM user convenience

Another difference is that the old image takes an env var where you can pass SCRAM users, and the image knows how to use them when formatting the storage. The apache images do not have this convenience.

Instead, we can start up the broker and then using an admin client connected to the anonymous listener, create the scram credentials.

[k-wall]

An update on this, we want to switch to apache for 4.1 (I believe we aren't going to release any more versions of ozangunalp/kafka-native?)

Yes, that's the direction. That's not immutable - we could put the effort into that project - as a stop gap - if we really wanted to, however, I think that would be the wrong thing to do at this point.

SASL not working on apache native image

Let's just use the ordinary non-native images to workaround this. The tests will take slightly longer to run, but that's tolerable I think. We can add a @ImageFlavour(NATIVE | JAVA) later. I don't think we want to think of Native images as a deal breaker.

It would be nice to encourage/assist a fix in the Kafka upstream.

apache images dont have SCRAM user convenience

Instead, we can start up the broker and then using an admin client connected to the anonymous listener, create the scram credentials.

I think that's the way to go. We want the test framework to use Kafka's public APIs.

[tombentley]

It would be nice to encourage/assist a fix in the Kafka upstream.

I spoke to Mickael and Luke about this. It's not trivial for Apache Kafka to get/build the infrasturcture for keeping their graal stuff up to do (apparently it involved running the system tests with a graal agent so it know what code paths get used...). So for now, switching to the non-native images seems like the best, more realistic option.

[robobario]

FYI we decided to depend on the JRE-based upstream apache images, lets leave this issue open though and we can revisit in future