How to debug - Message production error: MessageSizeTooLarge

[VitaliiNykyforov]

When the vector tries to send a message to Kafka we get an error like this:
Some(KafkaError (Message production error: MessageSizeTooLarge (Broker: Message size too large)))

I know how to fix this on kafka cluster side, but I want to debug this problem and understand which of the services trying to send so large massage. How I can catch this problem message? Thanks

PS. Before I tried to use /usr/bin/vector tap, but there are a lot of messages and difficult to identify it.

[jszwedko]

It would be slow, but you could use a remap transform with something like:

len = length(encode_json(.))
if len > <some number of bytes> {
  log(.)
}

To log messages where, when they are encoded as JSON, are bigger than N bytes. The length might not exactly match the Kafka message size (the encoding and event would need to be the same) but should hopefully help you find the large messages.

A slight twist on this would be to use this in a route transform to route the large messages to somewhere other than the kafka sink, for example a file sink.

[VitaliiNykyforov]

Ok, thanks. I thought maybe this can be solved as an option for example enable debug logging or something like that.