receiving duplicate messages in RabbitMQ version 4.1.0

[britishdan]

We've been working with RabbitMQ 3.12.12 (Erlang 25.3.2.7) for a year and have not received any duplicate messages (as far as we know).
We upgraded to 4.1.0 (Erlang 27.3.3) this week and have started receiving duplicate messages.

We're deployed on cloudamqp in Google Cloud.
I can't see what version of the plugin we're using.
We're using Exchange level deduplication.

We're using 2 Debezium servers, which each produce the same message to an exchange with the following config:

Type: x-message-deduplication
arguments:
  x-cache-persistence: disk
  x-cache-size: 1048576
  x-cache-ttl: 86400000
  durable: true

This exchange is bound to the main exchange, with routing key: #.

We're not sure how to debug this issue.
Maybe you can give us some direction on where to look.
Maybe someone else has also experienced this issue.

Any help is appreciated.

[gomoripeti]

Hi @britishdan

I think this issue is a duplicate of #121

As you run your RabbitMQ on CloudAMQP, if you send a support email to CloudAMQP and tell us which server is affected, we can help debug the issue (please reference this issue and ask help from the "CloudAMQP Erlang team")

let's continue the discussion about duplicate messages on 4.1.x in #121

[britishdan]

Ok. I did see that issue but since ours is on the Exchange level, I opened this issue.
I'll follow there and I'll work with support.
Thank you!

[noxdafox]

@gomoripeti this is not related to either #121 or #118 (duplicated ones). In those cases the issue is known and I am working for a fix. It will take a bit since I need to re-work the caching layer APIs.

This one might share the same root cause as #120. Since RabbitMQ is replacing Mnesia with Khepri, they are rolling out Mnesia bit by bit. The plugin used to rely on the broker management of Mnesia to implement its caching layer. We might need to onboard this function on the plugin itself.

@britishdan could you please confirm me whether you are still using Mnesia or you switched to Khepri for the metadata storage of your broker?
See: https://www.rabbitmq.com/docs/metadata-store/how-to-enable-khepri