You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello,
I'm currently exploring how to run MM2 with exactly once source connector support via the Strimzi KafkaMirrorMaker2 resource, but my knowledge about the connect framework is limited and the documentation feels a bit convoluted.
My current configuration looks like this:
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker2
metadata:
labels:
name: strimzi-mirrormaker
spec:
clusters:
- alias: source
bootstrapServers: 'source:9092'
- alias: target
bootstrapServers: 'target:9092'
config: # config for cluster specified in connectCluster gets forwarded to Kafka Connect config
exactly.once.source.support: enabled # This is important so Kafka Connect has support enabled and workers are started as ExactlyOnceWorkerSourceTask
connectCluster: target
mirrors:
- sourceCluster: source
targetCluster: target
checkpointConnector:
config:
consumer.isolation.level: read_committed # This might not be necessary
heartbeatConnector:
config:
consumer.isolation.level: read_committed # This might not be necessary
sourceConnector:
config:
consumer.isolation.level: read_committed # This is important so only committed messages are mirrored and only committed offsets in mirrormaker2-cluster-offsets are read
exactly.once.support: required # This might not be necessary
version: 3.5.1
Would this setup be sufficient and can I get rid of the marked configuration values?
And are there already plans to add this exactly once support as a configuration option directly in the resource? (I'm thinking about adding a flag that automatically sets up the required properties)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
I'm currently exploring how to run MM2 with exactly once source connector support via the Strimzi KafkaMirrorMaker2 resource, but my knowledge about the connect framework is limited and the documentation feels a bit convoluted.
My current configuration looks like this:
Would this setup be sufficient and can I get rid of the marked configuration values?
And are there already plans to add this exactly once support as a configuration option directly in the resource? (I'm thinking about adding a flag that automatically sets up the required properties)
Thanks for your help and insights!
Kay
Beta Was this translation helpful? Give feedback.
All reactions