Skip to content

Conversation

@torosent
Copy link
Member

@torosent torosent commented Dec 2, 2025

Summary

This PR adds a new configuration option IncludeInstanceIdInOperationName that allows users to include the Durable Orchestration Instance ID in the Application Insights operation name (operation_Name). This addresses a significant observability gap in the .NET Isolated worker model.

Problem

When using Durable Functions with Application Insights in the .NET Isolated worker model, all orchestrations appear under the same Operation Name in the Application Insights Failures tab. When multiple orchestrations fail simultaneously, failures are aggregated together, making it extremely difficult to diagnose which specific orchestration instance failed without manually drilling down into each sample.

Why TelemetryInitializer doesn't work in Isolated Model

In the in-process model, users could implement ITelemetryInitializer to modify the operation name because:

  • Function code runs in the same process as the Azure Functions host
  • The TelemetryInitializer can intercept all telemetry including RequestTelemetry

In the isolated worker model, this approach fails because:

  • Function code runs in a separate process from the host
  • The Durable Functions extension creates telemetry in the host process
  • ITelemetryInitializer in the worker process cannot intercept host-generated telemetry
  • The two processes communicate via gRPC, not shared memory
┌───────────────────────────────┐     ┌─────────────────────────────┐
│   Azure Functions Host        │     │   Worker Process            │
│         Process               │     │   (Your Code)               │
│                               │     │                             │
│  ┌─────────────────────────┐  │     │  ┌───────────────────────┐  │
│  │ Durable Extension       │  │     │  │  Your Function Code   │  │
│  │                         │  │     │  │                       │  │
│  │ Creates RequestTelemetry│  │◄───►│  │  ITelemetryInitializer│  │
│  │ HERE                    │  │  X  │  │  (Can't see host      │  │
│  │                         │  │     │  │   telemetry!)         │  │
│  └─────────────────────────┘  │     │  └───────────────────────┘  │
└───────────────────────────────┘     └─────────────────────────────┘
         gRPC communication

Solution

This PR modifies the telemetry at the source in the host extension, adding the instance ID to the operation name before it's sent to Application Insights.

Changes

  1. TraceOptions.cs - Added new configuration property:

    public bool IncludeInstanceIdInOperationName { get; set; } = false;
  2. Schema.cs - Added overloads to SpanNames methods that accept instance ID:

    • CreateOrchestration(name, version, instanceId, includeInstanceId)
    • CallOrSignalEntity(name, operation, instanceId, includeInstanceId)
    • EntityStartsAnOrchestration(name, instanceId, includeInstanceId)
  3. TraceHelper.cs - Updated StartActivityForNewOrchestration to accept and use the configuration

  4. WebJobsTelemetryModule.cs - Added setting Context.Operation.Name for V2 distributed tracing

  5. HttpApiHandler.cs & TaskHubGrpcServer.cs - Pass configuration when creating orchestration activities

  6. SchemaSpanNamesTests.cs - Added 15 unit tests

Usage

Users can enable this feature in host.json:

{
  "version": "2.0",
  "extensions": {
    "durableTask": {
      "tracing": {
        "DistributedTracingEnabled": true,
        "Version": "V2",
        "IncludeInstanceIdInOperationName": true
      }
    }
  }
}

Result

When enabled, the operation name changes from:

  • create_orchestration:MyOrchestration

To:

  • create_orchestration:MyOrchestration (abc123-def456-ghi789)

This allows Application Insights to group failures by individual orchestration instance.

Breaking Changes

None. This is a purely additive change:

  • New configuration option defaults to false
  • Original method signatures are preserved (new overloads added)
  • All modified APIs are internal
  • Existing behavior is unchanged unless explicitly opted-in

Issue describing the changes in this PR

resolves #3258

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation PR is ready to merge and referenced in pending_docs.md
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • I have added all required tests (Unit tests, E2E tests)
  • My changes do not require any extra work to be leveraged by OutOfProc SDKs
    • Otherwise: That work is being tracked here: #issue_or_pr_in_each_sdk
  • My changes do not change the version of the WebJobs.Extensions.DurableTask package
    • Otherwise: major or minor version updates are reflected in /src/Worker.Extensions.DurableTask/AssemblyInfo.cs
  • My changes do not add EventIds to our EventSource logs
    • Otherwise: Ensure the EventIds are within the supported range in our existing Windows infrastructure.
  • My changes should be added to v2.x branch.
    • Otherwise: This change applies exclusively to WebJobs.Extensions.DurableTask v3.x.

Fixes #3258

This change adds a new configuration option 'IncludeInstanceIdInOperationName'
that allows users to include the orchestration instance ID in the Application
Insights operation name (operation_Name) for better failure grouping and
debugging in the .NET Isolated worker model.

Changes:
- Add IncludeInstanceIdInOperationName option to TraceOptions
- Update Schema.SpanNames with overloads that support instance ID
- Update TraceHelper to pass instance ID when creating activities
- Update WebJobsTelemetryModule to set Operation.Name
- Add unit tests for Schema.SpanNames
@torosent torosent marked this pull request as ready for review December 3, 2025 22:23
@torosent torosent requested a review from bachuv December 3, 2025 22:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add durable orchestration InstanceId to Operation Name in Application Insights

2 participants