feat(csharp): add telemetry tags to driver activities (WI-6.2) by jadewang-db · Pull Request #177 · adbc-drivers/databricks

jadewang-db · 2026-01-22T18:42:26Z

🥞 Stacked PR

Use this link to review incremental changes.

stack/wi-3.5-metrics-aggregator [Files changed]
- stack/wi-3.5-circuit-breaker-manager [Files changed]
  - stack/e2e-feature-flag-cache-tests [Files changed]
    - stack/databricks-activity-listener [Files changed]
      - stack/circuit-breaker-telemetry-exporter [Files changed]
        
        stack/telemetry-client-manager-wi-3.2 [Files changed]
        
        stack/telemetry-client-wi-5.5 [Files changed]
        
        stack/telemetry-client-manager-e2e-wi-7 [Files changed]
        
        stack/telemetry-client-e2e-tests-wi-7-standalone [Files changed]
        
        stack/wi-6.1-databricks-connection-telemetry-integration [Files changed]
        stack/wi-6.2-telemetry-tags-driver-activities [Files changed]
        stack/wi-9-full-integration-e2e-tests [Files changed]

Implement MetricsAggregator that aggregates Activity data by statement_id and handles exception buffering with terminal vs retryable classification. Key features: - ProcessActivity extracts tags and aggregates by statement_id using ConcurrentDictionary<string, StatementTelemetryContext> - CompleteStatement emits aggregated TelemetryEvent - RecordException flushes terminal exceptions immediately - RecordException buffers retryable exceptions until CompleteStatement - FlushAsync exports when batch size or time interval reached - Uses TelemetryTagRegistry to filter tags - Creates TelemetryFrontendLog wrapper with workspace_id - All exceptions swallowed and logged at TRACE level Implementation details: - Connection events emit immediately (no aggregation needed) - Statement events aggregate until CompleteStatement is called - Timer-based periodic flush using System.Threading.Timer - Thread-safe aggregation using ConcurrentDictionary - Nested StatementTelemetryContext holds aggregated metrics and buffered exceptions per statement Test coverage: - 29 unit tests covering all exit criteria - Tests for exception handling, tag filtering, frontend log wrapping - End-to-end statement lifecycle tests Co-Authored-By: Claude <noreply@anthropic.com>

Implement CircuitBreakerManager as a singleton that manages circuit breakers per host. Each host gets its own circuit breaker instance for isolation, preventing one failing endpoint from affecting others. Key features: - Singleton pattern with GetInstance() method - Per-host circuit breaker isolation using ConcurrentDictionary - Thread-safe concurrent access - Case-insensitive host matching - Support for both default and custom configurations This follows the JDBC driver pattern in CircuitBreakerManager.java. Co-Authored-By: Claude <noreply@anthropic.com>

Update all CircuitBreaker tests to use the new constructor that takes a CircuitBreakerConfig object instead of individual parameters. Changes: - Add helper methods CreateDefaultConfig() and CreateConfig() for cleaner test setup - Replace `new CircuitBreaker()` with `new CircuitBreaker(CreateDefaultConfig())` - Replace `new CircuitBreaker(failureThreshold: N)` with `new CircuitBreaker(CreateConfig(failureThreshold: N))` - Replace `new CircuitBreaker(failureThreshold: N, timeout: T)` with `new CircuitBreaker(CreateConfig(failureThreshold: N, timeout: T))` - Add new tests for constructor with null config and config storage Co-Authored-By: Claude <noreply@anthropic.com>

Add comprehensive E2E tests for feature flag fetching from real Databricks endpoints and validate caching and reference counting behavior: - FeatureFlagCache_FetchFromRealEndpoint_ReturnsBoolean: Tests real endpoint - FeatureFlagCache_CachesValue_DoesNotRefetchWithinTTL: Validates caching - FeatureFlagCache_InvalidHost_ReturnsDefaultFalse: Tests error handling - FeatureFlagCache_RefCountingWorks_CleanupAfterRelease: Tests ref counting Additional tests cover: - Cache expiry and refetch behavior - Null/empty host handling - Unknown host behavior - Multiple hosts with independent ref counts - Concurrent reference counting thread safety - False value caching - Cancellation propagation Co-Authored-By: Claude <noreply@anthropic.com>

Rewrite E2E tests for FeatureFlagCache to use the actual production API: - Use MergePropertiesWithFeatureFlagsAsync() instead of non-existent IsTelemetryEnabledAsync() - Use HasContext(), TryGetContext(), RemoveContext(), Clear() for cache management testing - Remove tests for non-existent reference counting (RefCount property) - Remove tests for non-existent TelemetryEnabled, LastFetched, IsExpired - Fix namespace import from Telemetry to correct namespace - Add tests for TryGetHost protocol stripping and Uri extraction The previous tests were testing an API that didn't exist and would not compile. Co-Authored-By: Claude <noreply@anthropic.com>

Add DatabricksActivityListener that listens to 'Databricks.Adbc.Driver' ActivitySource, extracts metrics from activities, and delegates to MetricsAggregator. This implements Phase 5 of the telemetry design. Key features: - ShouldListenTo returns true for 'Databricks.Adbc.Driver' source - Sample callback respects feature flag (AllDataAndRecorded when enabled, None when disabled) - ActivityStopped callback delegates to MetricsAggregator.ProcessActivity - All callbacks wrapped in try-catch with TRACE logging - StopAsync flushes pending metrics via MetricsAggregator.FlushAsync - Supports dynamic feature flag checking via optional Func<bool> Co-Authored-By: Claude <noreply@anthropic.com>

Implement wrapper exporter that protects inner telemetry exporter with circuit breaker pattern. Key features: - Wraps ITelemetryExporter with circuit breaker protection - Uses CircuitBreakerManager.GetCircuitBreaker(host) for per-host isolation - Exports events when circuit is closed - Drops events silently when circuit is open (logged at DEBUG level) - Circuit breaker tracks failures BEFORE exceptions are swallowed This follows the design in Section 3.3 of the telemetry design document. Co-Authored-By: Claude <noreply@anthropic.com>

Implement per-host telemetry client management with reference counting to prevent rate limiting from concurrent connections. - ITelemetryClient: Interface for telemetry clients with ExportAsync and CloseAsync methods - TelemetryClientHolder: Holds client and reference count with atomic operations using Interlocked - TelemetryClientManager: Singleton factory managing one client per host using ConcurrentDictionary for thread-safety - TelemetryClientAdapter: Adapter bridging ITelemetryExporter to ITelemetryClient interface Key features: - GetInstance() returns singleton - GetOrCreateClient() creates/returns client and increments RefCount - ReleaseClientAsync() decrements RefCount, closes client when zero - Same host returns same client instance (case-insensitive) - Thread-safe with ConcurrentDictionary and atomic ref counting - All exceptions swallowed per telemetry design requirement Co-Authored-By: Claude <noreply@anthropic.com>

Implement TelemetryClient that coordinates listener, aggregator, and exporter. Manages background flush task and graceful shutdown. Changes: - Add TelemetryClient.cs implementing ITelemetryClient interface - Constructor initializes full telemetry pipeline: DatabricksTelemetryExporter → CircuitBreakerTelemetryExporter → MetricsAggregator → DatabricksActivityListener - ExportAsync delegates to the circuit breaker protected exporter - CloseAsync implements graceful shutdown per design doc Section 9.3: - Cancels background flush task - Stops listener (which flushes pending metrics) - Waits for background task with 5s timeout - Disposes all resources - All exceptions swallowed and logged at TRACE level - Background flush task periodically exports pending metrics - Update TelemetryClientManager.CreateClient() to use TelemetryClient - Add comprehensive unit tests (21 tests) covering: - Constructor initialization - Export delegation - Close with flush and cancellation - Exception swallowing during close - Background flush behavior - Thread safety Test file: csharp/test/Unit/Telemetry/TelemetryClientTests.cs Co-Authored-By: Claude <noreply@anthropic.com>

…ATE) Add end-to-end tests for TelemetryClientManager per-host client management: - TelemetryClientManager_SameHost_ReturnsSameClient: Validates same host returns same client - TelemetryClientManager_DifferentHosts_ReturnsDifferentClients: Validates different hosts get different clients - TelemetryClientManager_ConcurrentAccess_ThreadSafe: Validates thread-safety under concurrent access - TelemetryClientManager_LastRelease_ClosesClient: Validates reference counting and cleanup Additional tests for comprehensive coverage: - Case-insensitive host comparison - Mixed hosts handling - Concurrent release behavior - Concurrent get and release operations - Concurrent access to multiple hosts - Unknown host release handling - Exception swallowing during close Co-Authored-By: Claude <noreply@anthropic.com>

…2E GATE) Add comprehensive E2E tests for TelemetryClient in isolation against real Databricks endpoint. Tests validate: - Single event export to real endpoint - Batch event export to real endpoint - Authenticated endpoint usage (/telemetry-ext) - Circuit breaker opens after consecutive failures (default threshold 5) - Circuit breaker recovers after timeout period (half-open -> closed) - Graceful close flushes all pending events - Idempotent close behavior - Export after close doesn't throw The tests use mock exporters for circuit breaker behavior testing since the DatabricksTelemetryExporter swallows exceptions internally. This allows the CircuitBreakerTelemetryExporter wrapper to properly track failures. Co-Authored-By: Claude <noreply@anthropic.com>

Integrate telemetry components into DatabricksConnection lifecycle: - Initialize telemetry in HandleOpenSessionResponse after session is established - Check FeatureFlagCache.IsTelemetryEnabledAsync for server-side feature flag - Create TelemetryClient via TelemetryClientManager.GetOrCreateClient when enabled - Release telemetry resources in Dispose with proper reference counting - All telemetry exceptions are swallowed per requirement (Section 8.1) Implementation follows design doc sections 6.2 (Initialization) and 9.2 (Connection Close). Co-Authored-By: Claude <noreply@anthropic.com>

Add telemetry-specific tags to existing driver activities: Connection Activities: - driver.version, driver.os, driver.runtime - feature.cloudfetch, feature.lz4 Statement Activities: - result.format (inline/cloudfetch) in DatabricksCompositeReader - result.chunk_count, result.bytes_downloaded in CloudFetchDownloader - poll.count, poll.latency_ms in StatementExecutionStatement This implements Section 4.2 of telemetry-design.md for Activity Tags by Event Type, enabling Databricks telemetry service to track driver usage patterns and performance metrics. Co-Authored-By: Claude <noreply@anthropic.com>

jadewang-db mentioned this pull request Jan 22, 2026

test(csharp): add full integration E2E tests for telemetry (WI-9) #178

Closed

jadewang-db had a problem deploying to azure-prod January 22, 2026 18:42 — with GitHub Actions Error

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from d6e909c to 1813066 Compare January 22, 2026 22:24

jadewang-db had a problem deploying to azure-prod January 22, 2026 22:24 — with GitHub Actions Error

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from 1813066 to 1f7cde0 Compare January 23, 2026 21:14

jadewang-db had a problem deploying to azure-prod January 23, 2026 21:15 — with GitHub Actions Error

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from 1f7cde0 to 2364122 Compare January 23, 2026 21:43

jadewang-db had a problem deploying to azure-prod January 23, 2026 21:43 — with GitHub Actions Error

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from 2364122 to d07c210 Compare January 26, 2026 22:43

jadewang-db had a problem deploying to azure-prod January 26, 2026 22:43 — with GitHub Actions Error

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from d07c210 to 47c7ee7 Compare January 26, 2026 23:01

jadewang-db had a problem deploying to azure-prod January 26, 2026 23:01 — with GitHub Actions Error

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from b647e8c to 3f776ce Compare February 10, 2026 18:35

jadewang-db had a problem deploying to azure-prod February 10, 2026 18:35 — with GitHub Actions Error

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from 3f776ce to 7fef380 Compare February 11, 2026 04:41

jadewang-db had a problem deploying to azure-prod February 11, 2026 04:42 — with GitHub Actions Error

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from 7fef380 to 20145ab Compare February 17, 2026 23:08

jadewang-db had a problem deploying to azure-prod February 17, 2026 23:08 — with GitHub Actions Error

Jade Wang and others added 4 commits February 26, 2026 18:39

address comments

e76e259

Fix flaky test

a325755

change to use the new proto model

cded7cd

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from 20145ab to c16537e Compare February 26, 2026 18:54

jadewang-db had a problem deploying to azure-prod February 26, 2026 18:54 — with GitHub Actions Error

Jade Wang and others added 14 commits February 27, 2026 22:47

metric aggregator change after proto

2bf4693

address initial comments

bde22f7

jadewang-db force-pushed the stack/wi-6.2-telemetry-tags-driver-activities branch from c16537e to 35a4a01 Compare February 27, 2026 22:47

jadewang-db requested a deployment to azure-prod February 27, 2026 22:47 — with GitHub Actions Waiting

jadewang-db closed this Mar 4, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(csharp): add telemetry tags to driver activities (WI-6.2)#177

feat(csharp): add telemetry tags to driver activities (WI-6.2)#177
jadewang-db wants to merge 18 commits intomainfrom
stack/wi-6.2-telemetry-tags-driver-activities

jadewang-db commented Jan 22, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jadewang-db commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🥞 Stacked PR

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jadewang-db commented Jan 22, 2026 •

edited

Loading