Kafka mirror maker2 topic replication without source cluster name

[subash-subramaniyan]

Im using kafka cluster KafkaMirrorMaker2 yaml file to syn the cluster topics to secondary cluster, But it copying the topic with source cluster name , I need to replicate the index as it in source cluster to secondary cluster please help me.

eg: mytoipc ----> source.mytopic

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker2
metadata:
name: my-mm2-cluster
spec:
version: 3.2.0
replicas: 1
connectCluster: source
clusters:
- alias: source
bootstrapServers: 'source-kafka-plain-bootstrap.test.in:9092'
- alias: destination
bootstrapServers: 'destination-kafka-plain-bootstrap.test.in:9092'
config:
config.storage.replication.factor: -1
offset.storage.replication.factor: -1
status.storage.replication.factor: -1
mirrors:
- sourceCluster: source
targetCluster: destination
sourceConnector:
config:
replication.factor: 1
offset-syncs.topic.replication.factor: 1
sync.topic.acls.enabled: 'false'
replication.policy.class: org.strimzi.kafka.connect.mirror.IdentityReplicationPolicy
heartbeatConnector:
config:
heartbeats.topic.replication.factor: 1
checkpointConnector:
config:
checkpoints.topic.replication.factor: 1
topicsPattern: .*
groupsPattern: .*

[scholzj]

It is hard to say if your YAML is correct when it is not formatted properly. You can also compare it your self against the example (for your Strimzi version) -> there is IIRC an example for the replication with the same names.

[scholzj]

The YAMl looks good in general. So I' not sure what the problem is. Depending on the version of Strimzi you use, you can check the example and apply the differences: https://github.com/strimzi/strimzi-kafka-operator/blob/main/examples/mirror-maker/kafka-mirror-maker-2-custom-replication-policy.yaml

[subash-subramaniyan]

Thanks for the information ,In 3..2.3 version , Kafka and zookeeper pods are not running , If we change version 3.2 its working

[scholzj]

I'm not sure how that is related ... and without the configs and logs, I will anyway not be able to help.

[subash-subramaniyan]

No issue, I will try to deploy with latest version and update the status , Thanks for the continues support

[subash-subramaniyan]

@scholzj able to deploy with latest helm charts and topics syn properly , But active-active syn create multiple message both cluster,
Please advise on this

[scholzj]

You cannot have both. You either need to have the topics renamed and then you can use active-active mirroring. Or have the anmes kept and the. You should mirror only in one direction.

…

On Mon, Oct 17, 2022, 18:06 Subash S ***@***.***> wrote: @scholzj <https://github.com/scholzj> able to deploy with latest helm charts and topics syn properly , But active-active syn create multiple message both cluster, Please advise on this [image: image] <https://user-images.githubusercontent.com/93960578/196227353-cea33e6d-9f14-425f-b6a1-03feef4885ac.png> — Reply to this email directly, view it on GitHub <#7489 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLFOR2GKL2ZENEYEZ7E4MLWDV2O7ANCNFSM6AAAAAARG6V2PA> . You are receiving this because you were mentioned.Message ID: <strimzi/strimzi-kafka-operator/repo-discussions/7489/comments/3898080@ github.com>

[subash-subramaniyan]

Sure , Is it possible to delete attached topics and how to delete multiple topics in single shot

[scholzj]

Sorry, I don't follow the question

…

On Mon, Oct 17, 2022, 18:15 Subash S ***@***.***> wrote: Sure , Is it possible to delete attached topics and how to delete multiple topics in single shot [image: image] <https://user-images.githubusercontent.com/93960578/196229049-bffe74f2-2732-4807-ac48-bdc04ba349de.png> — Reply to this email directly, view it on GitHub <#7489 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLFOR76OMLBF372TTHNIJLWDV3RPANCNFSM6AAAAAARG6V2PA> . You are receiving this because you were mentioned.Message ID: <strimzi/strimzi-kafka-operator/repo-discussions/7489/comments/3898150@ github.com>