MirrorMaker2 with event hub as source and target

[MaayanFarchi]

Hey there,
I know it's possible to create mirror between kafka and eventhub and the other way around.
I was wondering if it is possible to create mirror between two different event hubs?
and if so will the following configuration be correct?

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker2
metadata:
name: my-mm2-cluster-from-eh
spec:
version: 2.7.0
replicas: 1
connectCluster: "dest-eventhub"
clusters:
- alias: "source-eventhub"
bootstrapServers: myehkafkasource11516.servicebus.windows.net:9093
config:
config.storage.replication.factor: 1
offset.storage.replication.factor: 1
status.storage.replication.factor: 1
consumer.connections.max.idle.ms: 180000
consumer.metadata.max.age.ms: 180000
authentication:
type: plain
username: $ConnectionString
passwordSecret:
secretName: sourceeventhubssecret
password: sourceeventhubspassword
- alias: "dest-eventhub"
bootstrapServers: myehkafkadest26776.servicebus.windows.net:9093
config:
config.storage.replication.factor: 1
offset.storage.replication.factor: 1
status.storage.replication.factor: 1
consumer.connections.max.idle.ms: 180000
consumer.metadata.max.age.ms: 180000
authentication:
type: plain
username: $ConnectionString
passwordSecret:
secretName: desteventhubssecret
password: desteventhubspassword
tls:
trustedCertificates: []
mirrors:
- sourceCluster: "source-eventhub"
targetCluster: "dest-eventhub"
sourceConnector:
config:
replication.factor: 1
offset-syncs.topic.replication.factor: 1
sync.topic.acls.enabled: "false"
heartbeatConnector:
config:
heartbeats.topic.replication.factor: 1
checkpointConnector:
config:
checkpoints.topic.replication.factor: 1
topicsPattern: "."
groupsPattern: "."

[scholzj]

I think it should work, but I'm not sure anyone ever tried it.

[ppatierno]

Yeah I did the demo about Kafka - EH but never tried EH to EH. Don't know in details how the Microsoft Kafka protocol implementation works and if it would need more, but you could try.

[MaayanFarchi]

When using MirrorMaker2 I got the following error in the mirror-maker-pod;

**│ 2021-10-04 10:47:08,626 WARN [AdminClient clientId=adminclient-11] Connection to node -1 (myehkafkasource11516.servicebus.windows.net/40.112.242.70:9093) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.

When using the old MirrorMaker with the following configuration, it solved the issue;

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker
metadata:
name: my-mirror-maker
spec:
version: 2.7.0
replicas: 1
consumer:
bootstrapServers: myehkafkasource11516.servicebus.windows.net:9093
groupId: my-source-group-id
authentication:
type: plain
username: $ConnectionString
passwordSecret:
secretName: sourceeventhubssecret
password: sourceeventhubspassword
tls:
trustedCertificates: []
producer:
bootstrapServers: myehkafkadest26776.servicebus.windows.net:9093
authentication:
type: plain
username: $ConnectionString
passwordSecret:
secretName: desteventhubssecret
password: desteventhubspassword
tls:
trustedCertificates: []
whitelist: ".*"

[MaayanFarchi]

hey @ppatierno and @scholzj any guess why MirrorMaker would work and not MirrorMaker2? maybe some configuration that I forgot?