[Question] How can I decrease the consumer lag for of Mirror Maker 2?

[msrijita18]

When the kafka cluster is being pumped a heavy load of 40 millions messages of size 20 kb, the consumer lag for partitions are going as high as 150 K which takes about 1.5 hrs to catch up for the MM2.
I saw this issue but its not that helpful.

[scholzj]

I guess sharing your current configuration might be a good start ;-)

[msrijita18]

I went through some resources, and found that making the no of replicas of MM2 equal to the no of partitions (=3) reduced the lag to as low as 0.

[scholzj]

Yeah, number of replicas and tasks was something I thought would be interesting to see from your configuration as playing with those might improve performance.

[msrijita18]

Yeah, these configs helped. Thanks.