Skip to content

πŸš€ [NEW] Telemetry SupportΒ #73

@Burgyn

Description

@Burgyn

🫴 Proposal

Add comprehensive telemetry support to TeaPie, enabling detailed performance monitoring and statistics collection for HTTP requests, retry operations, and overall collection execution through standard .NET telemetry APIs.
The goal is to provide users with actionable insights about their API test performance while maintaining seamless integration with external monitoring systems like Application Insights, OpenTelemetry, or custom telemetry providers.

Default In-Memory Implementation:

// In init.csx - telemetry enabled by default
tp.EnableTelemetry(); // Optional call - can be enabled by default

Application Insights Telemetry:

// In init.csx
tp.EnableTelemetry(options => {
    options.UseApplicationInsights("instrumentation-key");
    options.EnableDetailedTracing = true;
    options.SampleRate = 1.0; // 100% sampling for development
});

Custom Telemetry Provider:

// In init.csx  
tp.EnableTelemetry(options => {
    options.UseCustomProvider(new MyTelemetryProvider());
    options.TrackMetrics("http_requests", "retry_attempts", "test_duration");
    options.EnableConsoleOutput = false; // Disable default console summary
});

✨ Motivation

When running extensive API test collections, developers often need to answer questions like:

  • Which requests are consistently slow and need optimization?
  • How often do retries occur and what's their success rate?
  • What's the overall performance trend of the API under test?
  • Are there performance regressions between test runs?

Currently, TeaPie provides excellent test results and detailed logging, but lacks structured performance insights that could help:

  • Identify performance bottlenecks in API endpoints
  • Monitor retry patterns and optimize retry strategies
  • Track performance trends over time for regression detection
  • Generate reports for stakeholders about API reliability
  • Integrate with monitoring systems like Application Insights

This telemetry support would transform TeaPie from just a testing tool into a comprehensive API performance monitoring solution, providing valuable data for both development and operations teams without requiring any external tools for basic insights.

Example Console Output

β”Œβ”€β”€β”€ πŸ“Š Performance Summary ─────────────────────────────────────────────────────────────┐
β”‚                                                                                      β”‚
β”‚  🌐 HTTP: 47 requests β€’ 95.7% success rate β€’ 234ms avg β€’ 45ms-1.2s range              β”‚
β”‚  πŸ”„ Retry: 12 attempts β€’ 5 requests (10.6%) β€’ 83.3% success rate                       β”‚
β”‚  ⏱️  Tests: 23 cases β€’ 28.4s total β€’ 1.65 req/s throughput                             β”‚ 
β”‚                                                                                         β”‚
β”‚  πŸš€ Fastest: GET /api/users/123 (45ms)                                                 β”‚
β”‚  🐌 Slowest: POST /api/data/bulk-import (1.2s)                                         β”‚
β”‚  πŸ”„ Most retried: POST /api/orders/process (4x)                                        β”‚
β”‚                                                                                        β”‚
β”‚                                                                                        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestproposalProposal of some functionality

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions