Skip to content

Message

ppatierno edited this page May 2, 2016 · 6 revisions

The AMQP protocol has a well defined data type system and metadata for describing content other than an opaque body, which can be encoded using the same AMQP data type system or handled as raw data bytes. Apache Kafka doesn't have such rich features on transferring messages which are handled as raw bytes.

In order to translate AMQP messages between AMQP client and Apache Kafka, a MessageConverter interface is defined with following two methods :

  • toKafkaRecord : handles the conversion between an AMQP message to Kafka record;
  • toAmqpMessage : translated a Kafka record to an AMQP messages;

The message converter is pluggable through the message.convert property inside the bridge configuration file (bridge.properties).

The bridge provides a DefaultMessageConverter (as default) and a pluggable JsonMessageConverter.

DefaultMessageConverter

It's the simplest converter which works in the following way.

From AMQP message to Kafka record :

  • All properties, application properties, message annotations are lost. They are not encoded in any way in the Kafka record;
  • If partition and key are specified as message annotations, they are get in order to specify partition and key for topic destination in the Kafka record;
  • The AMQP body always handled as bytes and put inside the Kafka record value. The converter supports AMQP value and raw data/binary encoding;

From Kafka record to AMQP message :

  • No properties, application properties, message annotations are generated/filled;
  • Only partition, offset and key message annotations are filled from the Kafka record related information;
  • The AMQP body is encoded as raw data/binary from the corresponding Kafka record value;

JsonMessageConverter

TBD

Clone this wiki locally