TelemetryFlow GO SDK

Enterprise-grade Go SDK for TelemetryFlow - the observability platform that provides unified metrics, logs, and traces collection following OpenTelemetry standards.

---

🌟 Features

• 100% OTLP Compliant: Full OpenTelemetry Protocol implementation
• DDD Architecture: Domain-Driven Design with clear bounded contexts
• CQRS Pattern: Separate command and query responsibilities
• Easy Integration: Builder pattern and environment-based configuration
• Exemplars Support: Metrics-to-traces correlation for powerful debugging
• Service Graph Ready: Compatible with OTEL Collector servicegraph connector
• Enhanced gRPC Settings: Configurable keepalive, buffer sizes, and message limits
• TelemetryFlow Headers: Custom headers for collector authentication
• Code Generators: CLI tools for quick project setup
  • telemetryflow-gen: SDK integration generator
  • telemetryflow-restapi: DDD + CQRS RESTful API generator
• Docker Ready: Multi-stage Dockerfile and Docker Compose configurations
• Production Ready: Comprehensive error handling, retries, and batching
• Type Safe: Strong typing with compile-time safety
• Zero Dependencies (core): Minimal external dependencies in core SDK
• E2E Testing: Complete end-to-end testing infrastructure
• Extensible: Easy to extend for custom use cases

📦 Installation

go get github.com/telemetryflow/telemetryflow-go-sdk

🚀 Quick Start

1. Environment Variables Setup

Create a .env file:

TELEMETRYFLOW_API_KEY_ID=tfk_your_key_id_here
TELEMETRYFLOW_API_KEY_SECRET=tfs_your_secret_here
TELEMETRYFLOW_ENDPOINT=api.telemetryflow.id:4317
TELEMETRYFLOW_SERVICE_NAME=my-go-service
TELEMETRYFLOW_SERVICE_VERSION=1.1.1
TELEMETRYFLOW_SERVICE_NAMESPACE=telemetryflow
TELEMETRYFLOW_COLLECTOR_ID=my-collector-id
ENV=production

2. Initialize SDK

package main

import (
    "context"
    "log"

    "github.com/telemetryflow/telemetryflow-go-sdk/pkg/telemetryflow"
)

func main() {
    // Create client from environment variables
    client, err := telemetryflow.NewFromEnv()
    if err != nil {
        log.Fatal(err)
    }

    // Initialize the SDK
    ctx := context.Background()
    if err := client.Initialize(ctx); err != nil {
        log.Fatal(err)
    }
    defer client.Shutdown(ctx)

    // Your application code...
}

3. Send Telemetry

// Send a metric
client.IncrementCounter(ctx, "api.requests", 1, map[string]interface{}{
    "method": "GET",
    "status": 200,
})

// Send a log
client.LogInfo(ctx, "User logged in", map[string]interface{}{
    "user_id": "12345",
})

// Create a trace span
spanID, _ := client.StartSpan(ctx, "process-order", "internal", map[string]interface{}{
    "order_id": "67890",
})
defer client.EndSpan(ctx, spanID, nil)

🏗️ Architecture

The SDK follows Domain-Driven Design (DDD) and CQRS patterns:

pkg/telemetryflow/
├── domain/          # Domain layer (entities, value objects)
│   ├── credentials.go
│   └── config.go
├── application/     # Application layer (use cases, CQRS)
│   ├── commands.go
│   └── queries.go
├── infrastructure/  # Infrastructure layer (OTLP exporters)
│   ├── exporters.go
│   └── handlers.go
└── client.go        # Public API (interface layer)

Domain Layer

Core business entities and value objects:

• Credentials: Immutable API key pair with validation
• TelemetryConfig: Aggregate root containing all configuration

Application Layer

CQRS implementation:

• Commands: RecordMetricCommand, EmitLogCommand, StartSpanCommand, etc.
• Queries: GetMetricQuery, GetLogsQuery, GetTraceQuery, etc.
• Command/Query Buses: Route requests to appropriate handlers

Infrastructure Layer

Technical implementations:

• OTLPExporterFactory: Creates OTLP exporters (gRPC/HTTP)
• TelemetryCommandHandler: Handles telemetry commands
• OpenTelemetry SDK integration

📚 Usage Examples

Builder Pattern

client := telemetryflow.NewBuilder().
    WithAPIKey("tfk_...", "tfs_...").
    WithEndpoint("api.telemetryflow.id:4317").
    WithService("my-service", "1.0.0").
    WithEnvironment("production").
    WithGRPC().
    WithSignals(true, true, true). // metrics, logs, traces
    WithCustomAttribute("team", "backend").
    MustBuild()

Metrics

// Counter
client.IncrementCounter(ctx, "requests.total", 1, map[string]interface{}{
    "method": "POST",
    "endpoint": "/api/users",
})

// Gauge
client.RecordGauge(ctx, "memory.usage", 512.0, map[string]interface{}{
    "unit": "MB",
})

// Histogram
client.RecordHistogram(ctx, "request.duration", 0.25, "s", map[string]interface{}{
    "endpoint": "/api/orders",
})

Logs

// Info level
client.LogInfo(ctx, "Application started", map[string]interface{}{
    "version": "1.0.0",
    "port": 8080,
})

// Warning level
client.LogWarn(ctx, "High memory usage", map[string]interface{}{
    "usage_mb": 512,
    "threshold_mb": 400,
})

// Error level
client.LogError(ctx, "Database connection failed", map[string]interface{}{
    "error": "timeout",
    "host": "db.example.com",
})

Traces

// Start a span
spanID, err := client.StartSpan(ctx, "process-payment", "internal", map[string]interface{}{
    "payment_method": "credit_card",
    "amount": 99.99,
})

// Add events to span
client.AddSpanEvent(ctx, spanID, "validation.complete", map[string]interface{}{
    "valid": true,
})

// End span (with optional error)
client.EndSpan(ctx, spanID, nil)

🛠️ Code Generators

The SDK includes powerful code generators to bootstrap your integration.

SDK Generator (telemetryflow-gen)

Generates TelemetryFlow SDK integration code for existing projects.

# Install
go install github.com/telemetryflow/telemetryflow-go-sdk/cmd/generator@latest

# Initialize integration
telemetryflow-gen init \
    --project "my-app" \
    --service "my-service" \
    --key-id "tfk_..." \
    --key-secret "tfs_..."

# Generate examples
telemetryflow-gen example http-server
telemetryflow-gen example worker

RESTful API Generator (telemetryflow-restapi)

Generates complete DDD + CQRS RESTful API projects with Echo framework.

# Install
go install github.com/telemetryflow/telemetryflow-go-sdk/cmd/generator-restfulapi@latest

# Create new project
telemetryflow-restapi new \
    --name order-service \
    --module github.com/myorg/order-service \
    --db-driver postgres

# Add entities
telemetryflow-restapi entity \
    --name Order \
    --fields 'customer_id:uuid,total:decimal,status:string'

See Generator Documentation and RESTful API Generator for details.

🐳 Docker

Using Docker

# Run SDK generator
docker run --rm -v $(pwd):/workspace telemetryflow/telemetryflow-sdk:latest \
    init --project myapp --output /workspace

# Run RESTful API generator
docker run --rm -v $(pwd):/workspace telemetryflow/telemetryflow-sdk:latest \
    telemetryflow-restapi new --name myapi --output /workspace

Docker Compose

Development environment with full observability stack:

# Start development environment
docker-compose up -d

# Start with observability tools (Jaeger, Prometheus, Grafana)
docker-compose --profile full up -d

# Run E2E tests
docker-compose -f docker-compose.e2e.yml up --abort-on-container-exit

Generated Structure

your-project/
├── telemetry/
│   ├── init.go           # SDK initialization
│   ├── metrics/
│   │   └── metrics.go    # Metrics helpers
│   ├── logs/
│   │   └── logs.go       # Logging helpers
│   ├── traces/
│   │   └── traces.go     # Tracing helpers
│   └── README.md
├── .env.telemetryflow    # Configuration template
└── example_*.go          # Generated examples

🎯 Advanced Configuration

Protocol Selection

// gRPC (default, recommended)
config.WithProtocol(domain.ProtocolGRPC)

// HTTP
config.WithProtocol(domain.ProtocolHTTP)

Retry Configuration

config.WithRetry(
    true,                    // enabled
    3,                       // max retries
    5 * time.Second,        // backoff duration
)

Batch Settings

config.WithBatchSettings(
    10 * time.Second,       // batch timeout
    512,                     // max batch size
)

Signal Control

// Enable specific signals only
config.WithSignals(
    true,   // metrics
    false,  // logs
    true,   // traces
)

// Or use convenience methods
config.WithMetricsOnly()
config.WithTracesOnly()

Exemplars Support (v1.1.1+)

Exemplars enable metrics-to-traces correlation for powerful debugging:

// Enabled by default, disable if not needed
client := telemetryflow.NewBuilder().
    WithAutoConfiguration().
    WithExemplars(false).  // Disable exemplars
    MustBuild()

Service Namespace (v1.1.1+)

// Set service namespace for multi-tenant environments
client := telemetryflow.NewBuilder().
    WithAutoConfiguration().
    WithServiceNamespace("my-namespace").
    MustBuild()

Collector ID (v1.1.1+)

// Set collector ID for TelemetryFlow backend authentication
client := telemetryflow.NewBuilder().
    WithAutoConfiguration().
    WithCollectorID("my-unique-collector-id").
    MustBuild()

Custom Resource Attributes

config.
    WithCustomAttribute("team", "backend").
    WithCustomAttribute("region", "us-east-1").
    WithCustomAttribute("datacenter", "dc1")

🔒 Security

API Key Management

API keys should never be hardcoded. Use environment variables or secure secret management:

// ✅ Good: From environment
client, _ := telemetryflow.NewFromEnv()

// ✅ Good: From secret manager (example)
apiKey := secretManager.GetSecret("telemetryflow/api-key")
client, _ := telemetryflow.NewBuilder().
    WithAPIKey(apiKey.KeyID, apiKey.KeySecret).
    Build()

// ❌ Bad: Hardcoded
client, _ := telemetryflow.NewBuilder().
    WithAPIKey("tfk_hardcoded", "tfs_secret"). // DON'T DO THIS
    Build()

TLS/SSL

By default, the SDK uses secure connections. Only disable for testing:

// Production (default)
config.WithInsecure(false)

// Testing only
config.WithInsecure(true)

📊 Best Practices

1. Initialize Once: Create one client instance per application
2. Defer Shutdown: Always defer client.Shutdown(ctx)
3. Context Propagation: Pass context through your call chain
4. Attribute Cardinality: Limit high-cardinality attributes
5. Batch Configuration: Tune batch settings for your workload
6. Error Handling: Always check errors from telemetry calls
7. Graceful Shutdown: Allow time for final flush

func main() {
    client, _ := telemetryflow.NewFromEnv()
    ctx := context.Background()

    // Initialize
    if err := client.Initialize(ctx); err != nil {
        log.Fatal(err)
    }

    // Ensure graceful shutdown
    defer func() {
        shutdownCtx, cancel := context.WithTimeout(
            context.Background(),
            10*time.Second,
        )
        defer cancel()

        client.Flush(shutdownCtx)
        client.Shutdown(shutdownCtx)
    }()

    // Application code...
}

🧪 Testing

The SDK includes comprehensive tests organized by type:

tests/
├── unit/              # Unit tests for individual components
│   ├── domain/        # Credentials, Config tests
│   ├── application/   # Command/Query tests
│   ├── infrastructure/# Template, HTTP, Database tests
│   └── client/        # Client, Builder tests
├── integration/       # Cross-layer integration tests
├── e2e/               # End-to-end pipeline tests
│   └── testdata/      # E2E test fixtures (OTEL config, nginx)
├── mocks/             # Mock implementations
└── fixtures/          # Test fixtures

Running Tests

# Run all tests
go test ./...

# Run unit tests only
go test ./tests/unit/...

# Run with verbose output
go test -v ./...

# Run with coverage
go test -cover ./...

# Generate coverage report
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

# Run short tests (skip integration)
go test -short ./...

# Run e2e tests (requires environment setup)
TELEMETRYFLOW_E2E=true go test ./tests/e2e/...

Test Coverage Goals

Layer	Target
Domain	90%+
Application	85%+
Infrastructure	80%+
Client	85%+

📖 Documentation

• Quickstart Guide - Get started in 5 minutes
• Architecture Guide - SDK architecture with DDD/CQRS patterns
• API Reference - Complete API documentation
• SDK Generator - telemetryflow-gen CLI documentation
• RESTful API Generator - telemetryflow-restapi CLI documentation
• Examples - Sample applications and patterns
• Changelog - Version history and release notes
• TelemetryFlow Platform - Platform documentation

🤝 Contributing

Contributions are welcome! Please read our Contributing Guide for details.

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🆘 Support

• 📧 Email: [email protected]
• 💬 Slack: TelemetryFlow Community
• 📚 Docs: https://docs.telemetryflow.id
• 🐛 Issues: GitHub Issues

🙏 Acknowledgments

• OpenTelemetry for the excellent instrumentation standard
• All contributors who have helped shape this SDK

---

Built with ❤️ by the DevOpsCorner Indonesia