Some logs and exceptions not linked in distributed tracing

[adeliabt]

I implemented distributed tracing in my durable function following the steps here: https://github.com/Azure/azure-functions-durable-extension/blob/dev/samples/correlation-csharp/getting-started.md

I have a couple of questions:

• The activity function invocations don't appear anymore in the Activity Function's > Monitor > Invocations list. I suspect because the requests don't contain cloud_RoleName and operationName anymore
• Any logging that occur in the Orchestrator doesn't have the same operationId as its OrchestratorClient and Activity Functions. They don't appear in the end-to-end transaction logs. This makes it more tricky to view the orchestrator logs or exceptions that happen in the orchestrator.

Are these expected side effects? Known issues? Or do I need to configure something to get the Orchestrator logs linked?

Thanks

[ConnorMcMahon]

This is great feedback for the feature overall. We plan on giving Distributed Tracing some love in our next release (v2.5.0), and some of these feedback items may be worth filing as bugs/enhancements.

Adding @bachuv who is working on this feature area right now for this upcoming release.

[adeliabt]

thanks, good to know!

[bachuv]

@adeliabt, thank you for the feedback! Both of the points you brought up are known issues right now so I created #1804 and #1805 for tracking. The second point can be fixed if you add some additional code in the orchestrator function.

You can find an example of how to correlate orchestrator logs with the end-to-end transaction operationId in SimpleCorrelationDemo.cs from the FunctionAppCorrelation sample. Specifically, you can see that the orchestrator function creates a TraceTelemetry object, assigns the relevant operationIds, and then tracks it.

var trace = new TraceTelemetry( $"Activity Id: {correlationContext.TraceParent} ParentSpanId: {correlationContext.ParentSpanId}");
trace.Context.Operation.Id = correlationContext.TelemetryContextOperationId;
trace.Context.Operation.ParentId = correlationContext.TelemetryContextOperationParentId;
this.telemetryClient.Track(trace);

Let us know if you have any other questions!

[adeliabt]

Thanks @bachuv ! is v2.50 scheduled to be released on May 28th?

[bachuv]

Yes, v2.5.0 is scheduled to be released on May 28th.

[mauve]

Hi @bachuv, I am still hitting the correlation bug, the code sample above, do I need to replace all my calls to log.LogXXX() with the above snippet?

[bachuv]

Hi @mauve, yes the workaround in orchestrator functions is to use the code shown above instead of log.LogXXX() for distributed tracing.

[michaelelleby]

@bachuv This produces undesirable results when used with Eternal Orchestrations. Eternal Orchestrations causes Application Insights to correlate every orchestration with all previous orchestrations. I have created a sample project to show this issue. Search for Microsoft.ApplicationInsights.OperationContext in Application Insights to see the issue.