principal.builder.class

[sreejesh-radhakrishnan-db]

A basic Q, new to K8s , we want to implement the principal.builder.class , the example I got from from link (https://github.com/strimzi/strimzi-kafka-operator/blob/7b020f17e9bd93d5dcd26bbb9bbca64a7da696b8/documentation/api/io.strimzi.api.kafka.model.listener.KafkaListenerAuthenticationCustom.adoc) I am assuming SimplePrincipal.class is going to the class which I write for my use case, but not sure how kafka during broker setup will pick this up?

IS there any example which shows the pre-requisite to setup K8S such that we can use the config as below? Any help or link to understand how this setup is done will be really appreciated

spec:
kafka:
config:
principal.builder.class: SimplePrincipal.class

[scholzj]

You will have to build your class into a JAR and build your own container images with your custom JAR in the classpath.

[sreejesh-radhakrishnan-db]

I am clear on building the jar and creating a container image using jib. But not sure how KAFKA will use this image when I install the manifest. Or are you referring me rebuilding the kafka container image instead of using image provided in quay by srimzi? Really sorry beginner in K8S space, any documentation as to how to implement this will be a real help.

[scholzj]

I'm not sure you can build the image with jib or how you would do it. In general, you basically need to create a Dockerfile whihc starts with something like FROM quay.io/strimzi/kafka:0.28.0-kafka-3.1.0 (depending on what Kafka and Strimzi version you use) and inside the Dockerfile, add the jar into the Kafka's libs directory. You build it and push it into some registry. And then you configure the Cluster Operator to use this image => either in environment variables in the Cluster Operator configuration or in the Kafka custom resource in .spec.kafka.image.

[sreejesh-radhakrishnan-db]

understood. thank you

[sreejesh-radhakrishnan-db]

Not a java dev person by myself, can someone please give a check, if this will work for use case -

Current Principle = CN=[email protected],O= Test Bank ,DC=test,DC=com

want to trim to [email protected]

public class CustomPrincipalBuilder implements KafkaPrincipalBuilder {
@OverRide
public KafkaPrincipal build(AuthenticationContext authenticationContext) throws KafkaException {
try {
String[] split = null;

        if (authenticationContext instanceof SslAuthenticationContext) {
            SSLSession sslSession = ((SslAuthenticationContext) authenticationContext).session();
            try {
                split = sslSession.getPeerPrincipal().getName().split(",");    // [email protected],O= Test Bank ,DC=test,DC=com
                return new KafkaPrincipal(
                        KafkaPrincipal.USER_TYPE, split[0].split("=")[1]);   // user = [email protected] , kafka
            } catch (SSLPeerUnverifiedException e) {
                throw new IllegalArgumentException("Cannot use an unverified peer for authentication", e);
            }
        }
       else {
            throw new KafkaException("Failed to build CustomPrincipal. SSLAuthenticationContext is required.");
        }
    } catch (Exception ex) {
        throw new KafkaException("Failed to build CustomPrincipal due to: ", ex);
    }
}

}

[sreejesh-radhakrishnan-db]

@scholzj again a basic Q. sorry. when I copy the jar to library should I be doing this

COPY KafkaCustomBuilder-1.0-SNAPSHOT.jar libs/

or this way

COPY KafkaCustomBuilder-1.0-SNAPSHOT.jar /opt/kafka/libs/

[scholzj]

I guess the second is better?

[sreejesh-radhakrishnan-db]

second it is then :-)

[sreejesh-radhakrishnan-db]

HI All , getting this error after deploying the image with the principal.builder.class

Broker not starting with Error:

" Invalid value CommonNamePrincipalBuilder for configuration principal.builder.class: Class CommonNamePrincipalBuilder could not be found. "

As part of Debug I did check:
the JAR CommonNamePrincipalBuilder.jar is available in /opt/kafka/libs as I tried to deploy without principal.builder.class and did a ls -lte of location /opt/kafka/libs/ on my kafka broker pod and I can see the jar on the libs/ folder.

Also I have set CLASSPATH in Dockerfile like - ENV CLASSPATH /opt/kafka/libs/CommonNamePrincipalBuilder.jar

the config I am using to use the principle class in kafka manefest is like

kafka:
config:

  principal.builder.class: CommonNamePrincipalBuilder.class

What am I doing wrong? any help would be really appreciated

[scholzj]

You should probably not set any environment variables. also keep in mind that class name is not the same as JAR name. You have to use the name of the class including the package etc.

[sreejesh-radhakrishnan-db]

Ah okay- I will remove the ENV and give a try. the class name and JAR name is same in my case, not have package either. basically just the class inside a jar build using mvn.

[scholzj]

Normally you would always have some package ... but I have no idea if that could cause any issues or not.

[sreejesh-radhakrishnan-db]

@scholzj I tried with no ENV set but again same error.

2022-02-25T10:24:24.705566848Zorg.apache.kafka.common.config.ConfigException: Invalid value CommonNamePrincipalBuilder.class for configuration principal.builder.class: Class CommonNamePrincipalBuilder.class could not be found. at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:744) at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:490) at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:483) at org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:113) at org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:146) at kafka.server.KafkaConfig.(KafkaConfig.scala:1463) at kafka.server.KafkaConfig.(KafkaConfig.scala:1466) at kafka.Kafka$.buildServer(Kafka.scala:67) at kafka.Kafka$.main(Kafka.scala:87) at kafka.Kafka.main(Kafka.scala)

Any help on how I can debug this problem further ? please let me know.

On class - I am following similar way as what was on example hence did'nt package it up. I could give a try with packaging it up. but should I follow any specific kafka packing standard like org.kaka. or can I use usual package like any jar's?

as in
spec:
kafka:
config:
principal.builder.class: SimplePrincipal.class

[sreejesh-radhakrishnan-db]

tried with out the .class as well. also gave the absolute path, added the package and tried to access. its throwing same error. any idea how kafka set classpath or reference the jar in libs/ folder? just thinking if I have to explicitly call out my jar? not sure if the documentation missed any obvious step the implementer overlooked? Sorry just thinking aloud , without this working I cannot move my application to use strmzi. hence the push and urgency. appreciate any help on this topic.

[scholzj]

Everything in the libs directory should be in the classpath.

[sreejesh-radhakrishnan-db]

Not sure how to proceed. But as part of test.

a) I put my class directly in libs/ and referenced it in my kafka config . then as well getting same CLASS not found.

b) I randomly used a class which is present as part of another jar in libs/ directory and I am getting below error

equirement failed: principal.builder.class must implement KafkaPrincipalSerde at scala.Predef$

this make me feel, some how my jar and class is not getting added to the classpath. any idea which script I need to look into to understand how kafka set CP at all? may be devil is in the detail ;-)

[scholzj]

The Kafka scripts configure their own classpath. TBH, I'm not really sure what exactly are you doing. You say things such as I put my class in libs/. But as I said before, you cannot put the class there. You have to build a JAR and add the JAR there. If you think you did that right, you should also verify if it is actually in the image the broker is using or whether you are maybe using a wrong image or something.

[sreejesh-radhakrishnan-db]

sorry, let me clarify I am doing exactly this

create a Dockerfile which starts with something like FROM quay.io/strimzi/kafka:0.28.0-kafka-3.1.0 and inside the Dockerfile, added the jar into the Kafka's /opt/kafka/libs directory. I build it and push it into my image registry. And then configured the Cluster Operator to use this image in Cluster Operator configuration

I checked the image kafka is using its the image I build.

[sreejesh-radhakrishnan-db]

HI, got the class added to classpath and now the custom class I am passing is getting picked up. but I am getting the below error and broker is not starting

2022-02-28T17:12:45.005372133Zjava.lang.IllegalArgumentException: requirement failed: principal.builder.class must implement KafkaPrincipalSerde at scala.Predef$.require(Predef.scala:337) at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:2189) at kafka.server.KafkaConfig.(KafkaConfig.scala:1992) at kafka.server.KafkaConfig.(KafkaConfig.scala:1466) at kafka.Kafka$.buildServer(Kafka.scala:67) at kafka.Kafka$.main(Kafka.scala:87) at kafka.Kafka.main(Kafka.scala)

my class does implement as follows so not sure what I am doing wrong :

public class CustomCNPrincipalBuilder implements KafkaPrincipalBuilder
{
.......
}