otel sdk unexpectedly extract log data

[howan36]

{
"@timestamp": "2024-11-03T01:19:54.333781834+08:00",
"@Version": "1",
"message": "200 OK GET https://api-dev-mpp.xxxx.cn/api/routes/updatedAt",
"logger_name": "org.zalando.logbook.Logbook",
"thread_name": "reactor-http-epoll-4",
"level": "TRACE",
"level_value": 5000,
"http_request": Object{...},
"http_response": Object{...}
}

the log from my container stdout is like above, but data in grafana loki only has message field content,
like: timestape 200 OK POST https://api-dev-mpp.xxxx.cn/api/routes/updatedAt, others are missing

i am not sure how to change this default extract behaviour of java otel sdk

[trask]

are you using logstash? if so, I wonder if #12513 will help...

[howan36]

we use logback as logging framework

[trask]

can you give an example of how you are logging that ends up outputting the json above?

[howan36]

as my understanding, sdk will extract some message fields to attributes like trace_id, thread etc, but field http_request and http_response should be kept.

[howan36]

<property name="PATTERN"
          value="%d{yyyy-MM-dd HH:mm:ss.SSS} [trace_id=%X{trace_id:-},span_id=%X{span_id:-}] [%X{x-request-id}] [%thread] [scenario:%X{scenario}] [activityId:%X{activityId}] [partyUid:%X{partyUid}] [extendContext:%X{extendContext}] %-5level %cyan(%logger{64}) - %msg %n"/>

<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
        <layout class="com.ikea.common.spring.filter.http.CustomPatternLayout">
            <pattern>${PATTERN}</pattern>
        </layout>
    </encoder>
</appender>

<appender name="Async" class="ch.qos.logback.classic.AsyncAppender">
    <appender-ref ref="Console"/>
</appender>

<root level="INFO">
    <appender-ref ref="Async"/>
</root>

[howan36]

above is my log config

[trask]

what do you pass to the logger, e.g. message, mdc, etc?

[laurit]

@howan36 the log output and the configuration you have provided do no match as far as I can tell. Please provide a minimal application that reproduces your issue along with any necessary instructions.

[howan36]

we true use logstash markers, and when for that issuer PR will be merged

[trask]

I've tagged that PR to automatically close this issue, so you'll be notified when it's merged