-
Notifications
You must be signed in to change notification settings - Fork 4
Description
π«΄ Proposal
Add comprehensive telemetry support to TeaPie, enabling detailed performance monitoring and statistics collection for HTTP requests, retry operations, and overall collection execution through standard .NET telemetry APIs.
The goal is to provide users with actionable insights about their API test performance while maintaining seamless integration with external monitoring systems like Application Insights, OpenTelemetry, or custom telemetry providers.
Default In-Memory Implementation:
// In init.csx - telemetry enabled by default
tp.EnableTelemetry(); // Optional call - can be enabled by defaultApplication Insights Telemetry:
// In init.csx
tp.EnableTelemetry(options => {
options.UseApplicationInsights("instrumentation-key");
options.EnableDetailedTracing = true;
options.SampleRate = 1.0; // 100% sampling for development
});Custom Telemetry Provider:
// In init.csx
tp.EnableTelemetry(options => {
options.UseCustomProvider(new MyTelemetryProvider());
options.TrackMetrics("http_requests", "retry_attempts", "test_duration");
options.EnableConsoleOutput = false; // Disable default console summary
});β¨ Motivation
When running extensive API test collections, developers often need to answer questions like:
- Which requests are consistently slow and need optimization?
- How often do retries occur and what's their success rate?
- What's the overall performance trend of the API under test?
- Are there performance regressions between test runs?
Currently, TeaPie provides excellent test results and detailed logging, but lacks structured performance insights that could help:
- Identify performance bottlenecks in API endpoints
- Monitor retry patterns and optimize retry strategies
- Track performance trends over time for regression detection
- Generate reports for stakeholders about API reliability
- Integrate with monitoring systems like Application Insights
This telemetry support would transform TeaPie from just a testing tool into a comprehensive API performance monitoring solution, providing valuable data for both development and operations teams without requiring any external tools for basic insights.
Example Console Output
ββββ π Performance Summary ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β β
β π HTTP: 47 requests β’ 95.7% success rate β’ 234ms avg β’ 45ms-1.2s range β
β π Retry: 12 attempts β’ 5 requests (10.6%) β’ 83.3% success rate β
β β±οΈ Tests: 23 cases β’ 28.4s total β’ 1.65 req/s throughput β
β β
β π Fastest: GET /api/users/123 (45ms) β
β π Slowest: POST /api/data/bulk-import (1.2s) β
β π Most retried: POST /api/orders/process (4x) β
β β
β β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ