Properties in a log statement are shoved in to message in JsonEncoder

[mityakoval]

Hi!

I'm using a net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder and usually all the properties I add to my log statements are nicely displayed. One of the library's internal logs also seem to be in a JSON format but it's just getting formatted in to a string and shoved into the message property which is really hard to read.

{"app":"processor","time":"2024-07-31T15:12:31.301808321Z","logger":"com.azure.core.amqp.implementation.ReactorConnection","level":"INFO","thread":"reactor-executor-2","msg":"{\"az.sdk.message\":\"onConnectionShutdown. Shutting down.\",\"isTransient\":false,\"isInitiatedByClient\":false,\"shutdownMessage\":\"Finished processing pending tasks.\"}"}

Is there a way to instruct Logback to recognise the JSON logged by the library and merge according to the existing encoder config?

Here's the logback.xml:

<configuration>
    <contextName>processor</contextName>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <contextName>
                    <fieldName>app</fieldName>
                </contextName>
                <timestamp>
                    <fieldName>time</fieldName>
                    <timeZone>UTC</timeZone>
                </timestamp>
                <loggerName>
                    <fieldName>logger</fieldName>
                </loggerName>
                <logLevel>
                    <fieldName>level</fieldName>
                </logLevel>
                <threadName>
                    <fieldName>thread</fieldName>
                </threadName>
                <mdc/>
                <arguments>
                    <includeNonStructuredArguments>true</includeNonStructuredArguments>
                </arguments>
                <stackTrace>
                    <fieldName>stackTrace</fieldName>
                    <!-- maxLength - limit the length of the stack trace -->
                    <throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
                        <maxDepthPerThrowable>200</maxDepthPerThrowable>
                        <maxLength>14000</maxLength>
                        <rootCauseFirst>true</rootCauseFirst>
                    </throwableConverter>
                </stackTrace>
                <throwableClassName>
                    <fieldName>exceptionClass</fieldName>
                </throwableClassName>
                <message>
                    <fieldName>msg</fieldName>
                </message>
            </providers>
        </encoder>
    </appender>
    <root level="info">
        <appender-ref ref="STDOUT"/>
    </root>
    <logger name="org.eclipse.jetty" level="INFO"/>
    <logger name="io.netty" level="INFO"/>
    <logger name="io.ktor" level="INFO"/>
    <logger name="com.azure" level="INFO"/>
</configuration>

[philsttr]

Instead of the message provider, you can use the pattern provider with #tryJson{...}

<pattern>
    {
        "msg": "#tryJson{%message}"
    }
</pattern>

Note that this will add a lot of overhead to your logging, since it tries to format every message as JSON (and if it fails, it falls back to logging it as a string)

If high performance is a requirement, you could write your own custom JsonProvider implementation that optimizes it. For example, if you know a specific Logger always logs messages in proper JSON format, then you could do something like this:

public class MyJsonProvider extends AbstractJsonProvider<ILoggingEvent> {

    @Override
    public void writeTo(JsonGenerator generator, ILoggingEvent event) throws IOException {

        generator.writeFieldName("msg");

        if ("my.logger.that.writes.in.json".equals(event.getLoggerName())
                && event.getFormattedMessage().startsWith("{")) {

            generator.writeRaw(event.getFormattedMessage());

        } else {

            generator.writeString(event.getFormattedMessage());

        }
    }
}