Skip to content

Commit 3e14c50

Browse files
authored
fix Acrolinx issues
1 parent 95af100 commit 3e14c50

File tree

1 file changed

+12
-12
lines changed

1 file changed

+12
-12
lines changed

articles/hdinsight/kafka/kafka-mirrormaker-2-0-guide.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.date: 05/20/2022
99

1010
# How to use Kafka MirrorMaker 2.0 in data migration, replication and the use-cases
1111

12-
MirrorMaker 2 (MM2) is designed to make it easier to mirror or replicate topics from one Kafka cluster to another. It uses the Kafka Connect framework to simplify configuration and scaling. It dynamically detects changes to topics and ensures source and target topic properties are synchronized, including offsets and partitions.
12+
MirrorMaker 2.0 (MM2) is designed to make it easier to mirror or replicate topics from one Kafka cluster to another. It uses the Kafka Connect framework to simplify configuration and scaling. It dynamically detects changes to topics and ensures source and target topic properties are synchronized, including offsets and partitions.
1313

1414
In this article, you'll learn how to use Kafka MirrorMaker 2.0 in data migration/replication and the use-cases.
1515

@@ -27,7 +27,7 @@ Simulation of MirrorMaker 2.0 to replicate data points/offsets between two Kafka
2727

2828
### MM2 Internals
2929

30-
The Mirrormaker2 tool is composed of different connectors. These connectors are standard Kafka Connect connectors, which can be used directly with Kafka Connect in standalone or distributed mode.
30+
The MirrorMaker 2.0 tool is composed of different connectors. These connectors are standard Kafka Connect connectors, which can be used directly with Kafka Connect in standalone or distributed mode.
3131

3232
The summary of the broker setup process is as follows:
3333

@@ -51,8 +51,8 @@ The summary of the broker setup process is as follows:
5151

5252
### Deployment
5353

54-
1. Connect-mirror-maker.sh script bundled with the Kafka library implements a distributed MM2 cluster, which manages the Connect workers internally based on a config file. Internally Mirrormaker driver creates and handles pairs of each connector – MirrorSourceConnector, MirrorSinkConnector, MirrorCheckpoint connector and MirrorHeartbeatConnector.
55-
1. Start the Mirormaker 2.
54+
1. Connect-mirror-maker.sh script bundled with the Kafka library implements a distributed MM2 cluster, which manages the Connect workers internally based on a config file. Internally MirrorMaker driver creates and handles pairs of each connector – MirrorSourceConnector, MirrorSinkConnector, MirrorCheckpoint connector and MirrorHeartbeatConnector.
55+
1. Start MirrorMaker 2.0.
5656

5757
```
5858
./bin/connect-mirror-maker.sh ./config/mirror-maker.properties
@@ -64,7 +64,7 @@ The summary of the broker setup process is as follows:
6464
```
6565
export KAFKA_OPTS="-Djava.security.auth.login.config=<path-to-jaas.conf>"
6666
```
67-
### Sample Mirrormaker 2.0 Configuration file
67+
### Sample MirrorMaker 2.0 Configuration file
6868
6969
```
7070
# specify any number of cluster aliases
@@ -148,21 +148,21 @@ destination.sasl.mechanism=GSSAPI
148148

149149
**How will the consumers behave on migration, if that the destination cluster may have a different offset mapping to data points?**
150150

151-
Mirrormaker 2’s MirrorCheckpointConnector automatically stores consumer group offset checkpoints for consumer groups on the source cluster. Each checkpoint contains a mapping of the last committed offset for each group in the source cluster to the equivalent offset in destination cluster. So on migration the consumers that start consuming from same topic on the destination cluster will be able to resume receiving messages from the last offset they committed on the source cluster.
151+
The MirrorMaker 2.0 MirrorCheckpointConnector feature automatically stores consumer group offset checkpoints for consumer groups on the source cluster. Each checkpoint contains a mapping of the last committed offset for each group in the source cluster to the equivalent offset in destination cluster. So on migration the consumers that start consuming from same topic on the destination cluster will be able to resume receiving messages from the last offset they committed on the source cluster.
152152

153153
**How can we retain the exact topic name in destination cluster, as the source alias is prefixed with all the topics replicated?**
154154

155-
This is the default behavior in Mirrormaker2.0 to avoid data overriding in complex mirroring topologies. Customization of this needs to be done carefully in terms of replication flow design and topic management to avoid data loss. This can be done by using a custom replication policy class against “replication.policy.class”.
155+
This is the default behavior in MirrorMaker 2.0 to avoid data overriding in complex mirroring topologies. Customization of this needs to be done carefully in terms of replication flow design and topic management to avoid data loss. This can be done by using a custom replication policy class against “replication.policy.class”.
156156

157157
**Why do we see new internal topics created in my source and destination Kafka?**
158158

159-
Mirrormaker 2 internal topics are created by the Connectors to keep track of the replication process, monitoring, offset mapping and checkpointing.
159+
MirrorMaker 2.0 internal topics are created by the Connectors to keep track of the replication process, monitoring, offset mapping and checkpointing.
160160

161-
**Why does the mirrormaker creates only two replicas of the topic in the destination cluster while the source has more?**
161+
**Why does MirrorMaker create only two replicas of the topic in the destination cluster while the source has more?**
162162

163-
Mirrormaker 2 doesn’t replicate the replication factor of topics to target clusters. This can be controlled from MM2 config, by specifying the required number of “replication.factor”. The default value for the same is two.
163+
MirrormMker 2 doesn’t replicate the replication factor of topics to target clusters. This can be controlled from MM2 config, by specifying the required number of “replication.factor”. The default value for the same is two.
164164

165-
**How to use custom replication policy in Mirrormaker 2?**
165+
**How to use custom replication policy in MirrorMaker 2.0?**
166166

167167
Custom Replication Policy can be created by implementing the interface below.
168168

@@ -203,7 +203,7 @@ The implementation needs to be added to the Kafka classpath for the class refere
203203

204204
## References
205205

206-
[Mirrormaker2 Changes Apache Doc](https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0)
206+
[MirrorMaker 2.0 Changes Apache Doc](https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0)
207207

208208
[Client certificates setup for HDI Kafka](apache-kafka-ssl-encryption-authentication.md)
209209

0 commit comments

Comments
 (0)