Dealing with split logging lines

[gdevelek]

In the EFK stack, there appears to be a years-long-standing issue with large ( longer than 16KB) messages getting split into parts and appearing on Kibana in multiple lines. Such long messages typically include Java exception stack traces. The splitting makes parsing and therefore indexing impossible and messes things up completely for developers who need to read the logs.

By "message" I'm referring to the field with the label "message" that appears as part of the log entry that, of course, starts with the "timestamp" field.

As much as I have searched I have not found a FluentD filter that can identify and concatenate those parts and make them appear as a whole on a single log entry, where the JSON block can be properly parsed and indexed. I have tried a few filters of my own, with little success. Maintaining EFK is only a small part of my job so I'm not skilled with it at all.

Please help if you are aware of a solution.

Thanks