Movement of topic data using MirrorMaker 2 for different topics.

[Sharad-SKJ052]

Is it possible to use MirrorMaker 2 for the below scenario. -

1. Topic A is present in Kafka Cluster A. Topic B is created in Kafka Cluster B.
2. Setup MirrorMaker 2 between Cluster A and B and whitelist Topic A.
3. Question is, can the MirrorMaker replicate the data for Topic A into Topic B.

If that is not possible, is there any other way to implement the same use case.

[scholzj]

Normally, Mirror Maker either renames the topic by adding a prefix or does not rename it at all. The renaming is driven by the replication.policy.class option and the class it points to. If you want to rename the topics in some custom way, you would need to write your own policy for it. But not sure how easy it is to rename just one topic.

[AAR256]

I have a follow-up question on a similar point. Let's say if we're running MM2 to replicate topics from on-premises where the topic names are combination of upper and lower cases to Strimzi Kafka setup running on K8s cloud where uppercase resource names aren't allowed. Will MM2 will be able to resolve that by itself? If not, is there any other possible solution for this use case.

[scholzj]

I'm not sure I follow. Strimzi uses Apache Kafka binaries as provided by the Apache Kafka project. It does not fork Kafka or reimplement its APIs. So there is no Strimzi implementation of MM2 - it is the same software.

[AAR256]

Sure, that makes sense. I think the better question would be once we have created a custom replication policy class, how can we add it to the Strimzi MM2 instance?

[AAR256]

I found a reference template for custom policy MM2 configuration - https://github.com/strimzi/strimzi-kafka-operator/blob/main/examples/mirror-maker/kafka-mirror-maker-2-custom-replication-policy.yaml What would be the process to refer the custom policy created by a 3rd party in a similar way.

[scholzj]

You would need to write the policy, build it into a JAR and add it to the Strimzi container image by building a new one. For example through a new Dockerfile which you would build somewhere in your CI pipelines and then use.

[AAR256]

Got it. Thanks, that helps a ton. I'll get on it and will try to get that in place.