Load testing creating single trace instead of seperate traces

[rdoshi29]

Hi team,
I have a REST service in Java that has multiple spans in it, when I tested this REST service by hitting the URL multiple times with breaks in between, there were separate traces for each hit. But during load testing the traces are not being created properly, multiple hits have the same trace ID and for every new hit, whichever span is active at the moment becomes the parent for what was supposed to be a new trace. Attaching code to reproduce the same issue, and the heirarchy of spans on Jaeger UI
ApplicationSpi.java
`public class ApplicationSpi extends AbstractHandler{

static SpanExporter otlpExporter = OtlpGrpcSpanExporter.builder()
		.setEndpoint("http://localhost:4317/v1/traces")
		.build();
static SdkTracerProvider sdkTracerProvider = SdkTracerProvider.builder()
        .addSpanProcessor(BatchSpanProcessor.builder(otlpExporter).build())
                .setResource(
                        Resource.getDefault().toBuilder()
                            .put(ResourceAttributes.SERVICE_NAME, "OtelDemo")
                            .build())
                    .build();

static OpenTelemetry openTelemetry = OpenTelemetrySdk.builder()
        .setTracerProvider(sdkTracerProvider)
        .setPropagators(ContextPropagators.create(W3CTraceContextPropagator.getInstance()))
        .buildAndRegisterGlobal();

static Tracer tracerObj = openTelemetry.getTracer("instrumentation-library");

public static void main(String[] args) throws Exception {
	Server server = new Server(8083);
    server.setHandler(new ApplicationSpi());
        
    server.start();
    server.join();
	
}

public void handle(String target,
        Request baseRequest,
        HttpServletRequest request,
        HttpServletResponse response) 
throws IOException, ServletException
{
	Span pSpan = tracerObj.spanBuilder("Parent Span")
	        .startSpan();
	response.setContentType("text/html;charset=utf-8");
    response.setStatus(HttpServletResponse.SC_OK);
	response.getWriter().println("<h2>Hello</h2>");
	try{
	   ApplicationService appService = new ApplicationService();
	   appService.getChildSpan(tracerObj, pSpan.getSpanContext());
     }catch(Exception e) {
	   System.out.println(e);
   }finally {
	   pSpan.end();
   }
}

}
`

ApplicationService.java

`public class ApplicationService {

public void getChildSpan(Tracer tracerObj, SpanContext spanContext) {
	String traceID = spanContext.getTraceId();
	String spanID = spanContext.getSpanId();
	TraceFlags traceFlags = spanContext.getTraceFlags();
	TraceState traceState = spanContext.getTraceState();
	Scope scope = null;
	Span cSpan = null;
	Context context =
	        withSpanContext(
	            SpanContext.createFromRemoteParent(traceID, spanID, traceFlags, traceState),
	            Context.current());
	
	if(context != null) {
		scope = context.makeCurrent();
		
		cSpan = tracerObj.spanBuilder("Child span")
		        .startSpan();
		try {
			Thread.sleep((long)(Math.random() * 1000));
		} catch (InterruptedException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}finally {
			cSpan.end();
		}
		
	}
}

private static Context withSpanContext(SpanContext spanContext, Context context) {
    return context.with(Span.wrap(spanContext));
}

}
`

[jkwatson]

There are several things wrong with this instrumentation.

1. You must always close any Scope that you create, on the same thread that it was created.
2. If you use the standard idioms in your handle method, you won't need to do all the work you're doing in the "getChildSpan" method, since the span parenting will happen automatically.

I strongly recommend reading the documentation on manual instrumentation here, and making sure you understand what it happening around the management of the contexts and scope: https://opentelemetry.io/docs/instrumentation/java/manual/#create-nested-spans