Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ exporter/awss3exporter/ @open-telemetry
exporter/awsxrayexporter/ @open-telemetry/collector-contrib-approvers @wangzlei @srprash
exporter/azureblobexporter/ @open-telemetry/collector-contrib-approvers @hgaol @MovieStoreGuy
exporter/azuredataexplorerexporter/ @open-telemetry/collector-contrib-approvers @ag-ramachandran
exporter/azureeventhubsexporter/ @open-telemetry/collector-contrib-approvers
exporter/azuremonitorexporter/ @open-telemetry/collector-contrib-approvers @pcwiese @hgaol
exporter/bmchelixexporter/ @open-telemetry/collector-contrib-approvers @bertysentry @NassimBtk @MovieStoreGuy
exporter/cassandraexporter/ @open-telemetry/collector-contrib-approvers @atoulme @emreyalvac
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/beta_stability.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ body:
- exporter/awsxray
- exporter/azureblob
- exporter/azuredataexplorer
- exporter/azureeventhubs
- exporter/azuremonitor
- exporter/bmchelix
- exporter/carbon
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ body:
- exporter/awsxray
- exporter/azureblob
- exporter/azuredataexplorer
- exporter/azureeventhubs
- exporter/azuremonitor
- exporter/bmchelix
- exporter/carbon
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ body:
- exporter/awsxray
- exporter/azureblob
- exporter/azuredataexplorer
- exporter/azureeventhubs
- exporter/azuremonitor
- exporter/bmchelix
- exporter/carbon
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/other.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ body:
- exporter/awsxray
- exporter/azureblob
- exporter/azuredataexplorer
- exporter/azureeventhubs
- exporter/azuremonitor
- exporter/bmchelix
- exporter/carbon
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/unmaintained.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ body:
- exporter/awsxray
- exporter/azureblob
- exporter/azuredataexplorer
- exporter/azureeventhubs
- exporter/azuremonitor
- exporter/bmchelix
- exporter/carbon
Expand Down
1 change: 1 addition & 0 deletions exporter/azureeventhubsexporter/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
196 changes: 196 additions & 0 deletions exporter/azureeventhubsexporter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
# Azure Event Hubs Exporter

| Status | |
| ------------- |-----------|
| Stability | [alpha]: traces, metrics, logs |
| Distributions | [contrib] |
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Aexporter%2Fazureeventhubs%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aexporter%2Fazureeventhubs) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Aexporter%2Fazureeventhubs%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aexporter%2Fazureeventhubs) |
| Code Coverage | [![codecov](https://codecov.io/gh/open-telemetry/opentelemetry-collector-contrib/branch/main/graph/badge.svg?flag=azureeventhubsexporter)](https://codecov.io/gh/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/azureeventhubsexporter) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | \| Seeking more code owners! |

[alpha]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#alpha
[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
<!-- end autogenerated section -->

This exporter sends logs, traces and metrics to [Azure Event Hubs](https://docs.microsoft.com/azure/event-hubs/).

## Configuration

The following settings can be configured:

- `namespace` (required if not using connection_string): The Event Hubs namespace endpoint (e.g., "my-namespace.servicebus.windows.net")
- `event_hub`: Event Hub names for different telemetry types
- `logs` (default = `"logs"`): Event Hub name for logs
- `metrics` (default = `"metrics"`): Event Hub name for metrics
- `traces` (default = `"traces"`): Event Hub name for traces
- `auth`: Authentication configuration
- `type`: Authentication type. Supported values: `connection_string`, `service_principal`, `system_managed_identity`, `user_managed_identity`, `workload_identity`, `default_credentials`
- `connection_string`: Connection string to the Event Hubs namespace or Event Hub (required when type is `connection_string`)
- `tenant_id`: Tenant ID for Azure AD authentication (required for `service_principal` and `workload_identity`)
- `client_id`: Client ID (required for `service_principal`, `user_managed_identity`, and `workload_identity`)
- `client_secret`: Client secret (required for `service_principal`)
- `federated_token_file`: Path to federated token file (required for `workload_identity`)
- `format` (default = `"json"`): Format of encoded telemetry data. Supported values: `json`, `proto`
- `partition_key`: Partition key configuration for Event Hub partitioning
- `source`: How the partition key is generated. Options: `static`, `resource_attribute`, `trace_id`, `span_id`, `random`
- `value`: Used when source is `static` or specifies the attribute name when source is `resource_attribute`
- `max_event_size` (default = 1048576): Maximum size of an event in bytes (max: 1MB for Event Hubs)
- `batch_size` (default = 100): Number of events to batch before sending
- `retry_on_failure`: Retry configuration
- `enabled` (default = true): Whether to retry on failure
- `initial_interval` (default = 5s): Initial retry interval
- `max_interval` (default = 30s): Maximum retry interval
- `max_elapsed_time` (default = 5m): Maximum elapsed time for retries

## Examples

Using Connection String:

```yaml
exporters:
azureeventhubs:
auth:
type: connection_string
connection_string: "Endpoint=sb://my-namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=my-key"
event_hub:
logs: "otel-logs"
metrics: "otel-metrics"
traces: "otel-traces"
format: json
partition_key:
source: trace_id
max_event_size: 1048576
batch_size: 100
```

Using Entra ID Service Principal:

```yaml
exporters:
azureeventhubs:
namespace: "my-namespace.servicebus.windows.net"
auth:
type: service_principal
tenant_id: "your-tenant-id"
client_id: "your-client-id"
client_secret: "your-client-secret"
event_hub:
logs: "otel-logs"
metrics: "otel-metrics"
traces: "otel-traces"
format: proto
partition_key:
source: resource_attribute
value: "service.name"
```

Using Managed Identity (system-assigned):

```yaml
exporters:
azureeventhubs:
namespace: "my-namespace.servicebus.windows.net"
auth:
type: system_managed_identity
event_hub:
logs: "otel-logs"
metrics: "otel-metrics"
traces: "otel-traces"
format: json
partition_key:
source: random
```

Using Managed Identity (user-assigned):

```yaml
exporters:
azureeventhubs:
namespace: "my-namespace.servicebus.windows.net"
auth:
type: user_managed_identity
client_id: "your-managed-identity-client-id"
event_hub:
logs: "otel-logs"
metrics: "otel-metrics"
traces: "otel-traces"
```

Using Workload Identity (for Kubernetes):

```yaml
exporters:
azureeventhubs:
namespace: "my-namespace.servicebus.windows.net"
auth:
type: workload_identity
tenant_id: "your-tenant-id"
client_id: "your-client-id"
federated_token_file: "/var/run/secrets/azure/tokens/azure-identity-token"
event_hub:
logs: "otel-logs"
metrics: "otel-metrics"
traces: "otel-traces"
```

Using DefaultAzureCredential:

```yaml
exporters:
azureeventhubs:
namespace: "my-namespace.servicebus.windows.net"
auth:
type: default_credentials
event_hub:
logs: "otel-logs"
metrics: "otel-metrics"
traces: "otel-traces"
```

## Data Export

This exporter sends telemetry data to Azure Event Hubs with support for different data formats and partitioning strategies.

### Data Formats

The exporter supports two data formats:

- `json`: Telemetry data is encoded as JSON (default)
- `proto`: Telemetry data is encoded using Protocol Buffers

### Partition Key Strategies

The exporter supports different partition key strategies to control how data is distributed across Event Hub partitions:

- `static`: Uses a fixed partition key value
- `resource_attribute`: Uses the value of a specified resource attribute
- `trace_id`: Uses the trace ID from the telemetry data (traces and logs only)
- `span_id`: Uses the span ID from the telemetry data (traces only)
- `random`: Generates a random partition key for even distribution across partitions

### Event Hub Routing

By default, telemetry data is routed to different Event Hubs based on signal type:

- Logs → `logs` Event Hub
- Metrics → `metrics` Event Hub
- Traces → `traces` Event Hub

These can be customized using the `event_hub` configuration options.

## Authentication

The exporter supports multiple authentication methods:

- `connection_string`: Uses a connection string to authenticate (recommended for development)
- `service_principal`: Uses Azure AD service principal authentication
- `system_managed_identity`: Uses system-assigned managed identity
- `user_managed_identity`: Uses user-assigned managed identity
- `workload_identity`: Uses workload identity (for Kubernetes environments)
- `default_credentials`: Uses DefaultAzureCredential which tries multiple authentication methods

## Requirements

- Event Hubs must exist before starting the collector
- The authentication principal must have "Azure Event Hubs Data Sender" role on the Event Hubs
- Maximum event size is 1MB for Event Hubs Standard tier
51 changes: 51 additions & 0 deletions exporter/azureeventhubsexporter/client.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package azureeventhubsexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/azureeventhubsexporter"

import (
"context"

"github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs"
)

// eventDataBatch is an interface that defines the methods needed from azeventhubs.EventDataBatch
// This interface allows for mocking in tests
type eventDataBatch interface {
AddEventData(eventData *azeventhubs.EventData, options *azeventhubs.AddEventDataOptions) error
NumEvents() int32
}

// eventHubProducerClient is an interface that defines the methods needed from azeventhubs.ProducerClient
// This interface allows for mocking in tests
type eventHubProducerClient interface {
NewEventDataBatch(ctx context.Context, options *azeventhubs.EventDataBatchOptions) (eventDataBatch, error)
SendEventDataBatch(ctx context.Context, batch eventDataBatch, options *azeventhubs.SendEventDataBatchOptions) error
Close(ctx context.Context) error
}

// azureEventHubProducerClientWrapper wraps the Azure SDK ProducerClient to implement our interface
type azureEventHubProducerClientWrapper struct {
client *azeventhubs.ProducerClient
}

func (w *azureEventHubProducerClientWrapper) NewEventDataBatch(ctx context.Context, options *azeventhubs.EventDataBatchOptions) (eventDataBatch, error) {
return w.client.NewEventDataBatch(ctx, options)
}

func (w *azureEventHubProducerClientWrapper) SendEventDataBatch(ctx context.Context, batch eventDataBatch, options *azeventhubs.SendEventDataBatchOptions) error {
// Cast back to concrete type for Azure SDK
azureBatch, ok := batch.(*azeventhubs.EventDataBatch)
if !ok {
// For testing, just return nil as mock will handle it
return nil
}
return w.client.SendEventDataBatch(ctx, azureBatch, options)
}

func (w *azureEventHubProducerClientWrapper) Close(ctx context.Context) error {
return w.client.Close(ctx)
}

// Ensure that *azeventhubs.EventDataBatch implements eventDataBatch
var _ eventDataBatch = (*azeventhubs.EventDataBatch)(nil)
Loading