Option to convert log events to span events

[anuraaga]

Is your feature request related to a problem? Please describe.

It's not uncommon to be running only a tracing backend or otherwise have no way to ingest logs, for example for sites using jaeger. Then events cannot be consumed at all.

Describe the solution you'd like

An option that will have the logs sdk instead of queueing for OTLP export, check if there is an active span with the same context as the log. If true, convert to a span event. If not, drop it. I guess it would basically be a special logprocessor.

Describe alternatives you've considered

A collector plugin could be nice for being cross language but ordering challenges of spans coming in before logs would require some tricky time-based buffering (wait x seconds before assuming the span's events have all come in) that at best would delay telemetry and at worst would miss telemetry.

Additional context

Well, the answer to this topic will likely be needing a spec issue, but given how much engineering time has already been spent on this topic there, I'm putting it forth to more practical minded folks 😉

[codefromthecrypt]

I like doing this at SDK layer and having an ENV to enable it. This is much better than having each instrumentation make a similar modification.

---

To your point on the collector, some notes discussed in various places limit the viability for some reasons.

• processor requires a context to work with
• A context (datapoint, span event, etc.) is later used to apply the transformation, but the processor cannot create new contexts

So, you can't do cross signal processing. The way out suggested for "cross signal" stuff in the collector was a connector component, but I've not seen one built.

[jack-berg]

An option that will have the logs sdk instead of queueing for OTLP export, check if there is an active span with the same context as the log. If true, convert to a span event. If not, drop it. I guess it would basically be a special logprocessor.

I've got a prototype (draft) PR of this in #6650: https://github.com/open-telemetry/opentelemetry-java/pull/6650/files#diff-9a020a19daa2bf404ef87308c5948ee99a157bd3c68195dface5b2813ef30f04

The key bit is defining how log record events translate to span events given that log record bodies are AnyValue while span events are limited to standard attributes. I think long term the spec / proto will solve this possibly by extending span event with an AnyValue body, but I / we don't want to wait 🙂. My PR is pretty naive about this now and just records the logRecord.getBody().asString() to a span event attribute called body. (See this test to see how the Value#asString() works for different types.) With just a little effort, I could do a proper JSON encoding of the data.

What do folks think about landing a LogRecordProcessor like this in opentelemetry-java-contrib? It would be subject to change since I'm sure the spec will have something to say about this eventually, but this would provide a working implementation now and potentially influence / speed up the spec work.

[trask]

With just a little effort, I could do a proper JSON encoding of the data.

this would be nice ❤️

this would provide a working implementation now and potentially influence / speed up the spec work.

👍

[anuraaga]

Thanks @jack-berg! Searched existing issues but didn't think to search PRs, good to see something there already. I think having that available as a prototype would be great

[codefromthecrypt]

I think long term the spec / proto will solve this possibly by extending span event with an AnyValue body, but I / we don't want to wait

I agree with this approach long term, btw. We should level the playing field where people don't grasp and use an inconvenient signal type, just to access AnyValue which could be in another. 👍 towards a fair game ⚾!

[jack-berg]

PR here open-telemetry/opentelemetry-java-contrib#1551