Creating new "child" traces in process

[joachimm]

Due to size limits of traces in our Grafana/Tempo setup, I need to split traces into multiple smaller ones.

            for (Batch batch : batches) {
                SpanContext parentSpanContext = Span.current().getSpanContext();
                String parentTraceId = parentSpanContext.getTraceId();

                SpanContext childSpanContext = SpanContext.create(IdGenerator.random().generateTraceId(),
                        IdGenerator.random().generateSpanId(), TraceFlags.getDefault(), TraceState.getDefault());
                Context childContext = Context.current().with(Span.wrap(childSpanContext));
                Tracer tracer = GlobalOpenTelemetry.getTracer("batches");

                Span span = tracer.spanBuilder("download").setSpanKind(SpanKind.SERVER).setParent(childContext)
                        .addLink(parentSpanContext).startSpan();

                try (Scope unused = span.makeCurrent()) {
                                   // ... business logic here

Unfortunately when trying to find the "child" traces in Tempo, they do not seem to have been exported. The other traces are there as expected.
Anyone have any input in how this can be made to work?

[jkwatson]

what happens at the end of the try block? Are you ending your spans in the finally?

[joachimm]

Yes. like this.

                } catch (Exception e) {
                    span.setStatus(StatusCode.ERROR);
                    throw e;
                } finally {
                    span.end();
                }

[jkwatson]

I'm not sure what's going wrong here, but one thing you can simplify is to use spanBuilder.setNoParent() rather than artificially constructing a parent for the span you're building.

so...

Span span = tracer.spanBuilder("download").setSpanKind(SpanKind.SERVER).setNoParent()
                        .addLink(parentSpanContext).startSpan();

I don't know if this will change anything, but it will at least simplify your code a bit, which might make it easier to track down what's going on.

It results in:

Tracer tracer = GlobalOpenTelemetry.getTracer("batches");

for (Batch batch : batches) {
                SpanContext parentSpanContext = Span.current().getSpanContext();
 
                Span span = tracer.spanBuilder("download").setSpanKind(SpanKind.SERVER).setNoParent()
                        .addLink(parentSpanContext).startSpan();

                try (Scope unused = span.makeCurrent()) {
                                   // ... business logic here
                } catch (Exception e) {
                    span.setStatus(StatusCode.ERROR);
                    throw e;
                } finally {
                    span.end();
                }
}

If you use the logging exporter, can you see the spans that would be exported? Do they look like you expect?

I see you're calling GlobalOpenTelemetry to get your tracer...I assume that you're either a) using the javaagent, or b) have explicitly configured the global somewhere?

[joachimm]

Thanks, using the code you wrote it now works, everything is exported correctly. I am using java-agent btw.