Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

otel sdk unexpectedly extract log data #12573

Open
howan36 opened this issue Nov 5, 2024 · 10 comments · May be fixed by #12513
Open

otel sdk unexpectedly extract log data #12573

howan36 opened this issue Nov 5, 2024 · 10 comments · May be fixed by #12513

Comments

@howan36
Copy link

howan36 commented Nov 5, 2024

{
"@timestamp": "2024-11-03T01:19:54.333781834+08:00",
"@Version": "1",
"message": "200 OK GET https://api-dev-mpp.xxxx.cn/api/routes/updatedAt",
"logger_name": "org.zalando.logbook.Logbook",
"thread_name": "reactor-http-epoll-4",
"level": "TRACE",
"level_value": 5000,
"http_request": Object{...},
"http_response": Object{...}
}

the log from my container stdout is like above, but data in grafana loki only has message field content,
like: timestape 200 OK POST https://api-dev-mpp.xxxx.cn/api/routes/updatedAt, others are missing

i am not sure how to change this default extract behaviour of java otel sdk

@trask
Copy link
Member

trask commented Nov 5, 2024

are you using logstash? if so, I wonder if #12513 will help...

@howan36
Copy link
Author

howan36 commented Nov 6, 2024

we use logback as logging framework

@trask
Copy link
Member

trask commented Nov 6, 2024

can you give an example of how you are logging that ends up outputting the json above?

@howan36
Copy link
Author

howan36 commented Nov 6, 2024

as my understanding, sdk will extract some message fields to attributes like trace_id, thread etc, but field http_request and http_response should be kept.

@howan36
Copy link
Author

howan36 commented Nov 6, 2024

<property name="PATTERN"
          value="%d{yyyy-MM-dd HH:mm:ss.SSS} [trace_id=%X{trace_id:-},span_id=%X{span_id:-}] [%X{x-request-id}] [%thread] [scenario:%X{scenario}] [activityId:%X{activityId}] [partyUid:%X{partyUid}] [extendContext:%X{extendContext}] %-5level %cyan(%logger{64}) - %msg %n"/>

<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
        <layout class="com.ikea.common.spring.filter.http.CustomPatternLayout">
            <pattern>${PATTERN}</pattern>
        </layout>
    </encoder>
</appender>

<appender name="Async" class="ch.qos.logback.classic.AsyncAppender">
    <appender-ref ref="Console"/>
</appender>

<root level="INFO">
    <appender-ref ref="Async"/>
</root>

@howan36
Copy link
Author

howan36 commented Nov 6, 2024

above is my log config

@trask
Copy link
Member

trask commented Nov 6, 2024

what do you pass to the logger, e.g. message, mdc, etc?

@laurit
Copy link
Contributor

laurit commented Nov 6, 2024

@howan36 the log output and the configuration you have provided do no match as far as I can tell. Please provide a minimal application that reproduces your issue along with any necessary instructions.

@laurit laurit added needs author feedback Waiting for additional feedback from the author needs repro labels Nov 6, 2024
@howan36
Copy link
Author

howan36 commented Nov 6, 2024

we true use logstash markers, and when for that issuer PR will be merged

@github-actions github-actions bot removed the needs author feedback Waiting for additional feedback from the author label Nov 6, 2024
@trask
Copy link
Member

trask commented Nov 7, 2024

I've tagged that PR to automatically close this issue, so you'll be notified when it's merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants