Skip to content

Commit a756f00

Browse files
Copilottrask
Copilot
and
trask
committed
Add comprehensive telemetry testing using MockServer and OTLP validation
Co-authored-by: trask <[email protected]>
1 parent a046413 commit a756f00

File tree

5 files changed

+490
-0
lines changed

5 files changed

+490
-0
lines changed

‎reference-application/E2E-TEST.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,16 @@ The test verifies:
6060
- Log correlation
6161
- OTLP export to collector
6262

63+
### Telemetry Testing
64+
In addition to end-to-end infrastructure testing, the application includes **telemetry testing** that validates actual OpenTelemetry data export:
65+
66+
- **Trace validation**: Verifies spans are created with correct names, attributes, and events
67+
- **Metric validation**: Confirms custom metrics are exported properly
68+
- **Baggage testing**: Validates cross-cutting concern propagation
69+
- **MockServer integration**: Captures OTLP requests for detailed analysis
70+
71+
These tests run with the OpenTelemetry Java Agent and use the same protobuf parsing as the telemetry-testing example.
72+
6373
### Infrastructure
6474
- OpenTelemetry Collector OTLP ingestion
6575
- Prometheus metrics scraping

‎reference-application/README.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,10 +138,24 @@ All logs include:
138138

139139
### Testing
140140

141+
The reference application includes comprehensive testing:
142+
143+
#### Unit Tests
141144
```shell
142145
../gradlew test
143146
```
144147

148+
The test suite includes:
149+
- **Functional tests**: Verify all endpoints return correct responses
150+
- **Telemetry tests**: Validate OpenTelemetry data export using MockServer
151+
- Traces: HTTP spans, custom spans, span attributes, and events
152+
- Metrics: Custom counters and timers
153+
- Baggage: Cross-cutting concern propagation
154+
155+
The telemetry tests use MockServer to capture OTLP requests and verify that the application correctly generates and exports telemetry data for different scenarios.
156+
157+
For detailed information about telemetry testing, see [TELEMETRY-TESTING.md](TELEMETRY-TESTING.md).
158+
145159
### End-to-End Testing
146160

147161
Run the comprehensive end-to-end test that verifies the complete OpenTelemetry stack:

‎reference-application/TELEMETRY-TESTING.md‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# Telemetry Testing
2+
3+
This document explains the telemetry testing approach used in the reference application to validate OpenTelemetry data export.
4+
5+
## Overview
6+
7+
The `TelemetryTest` class demonstrates how to test that your application correctly generates and exports OpenTelemetry telemetry data. This approach is based on the [telemetry-testing example](../telemetry-testing) in the repository.
8+
9+
## How it Works
10+
11+
### MockServer Setup
12+
- Uses [MockServer](https://www.mock-server.com/) to simulate an OTLP collector
13+
- Listens on port 4318 (default OTLP HTTP endpoint)
14+
- Captures all OTLP requests sent by the OpenTelemetry Java Agent
15+
16+
### Test Configuration
17+
The test suite is configured to:
18+
- Run with OpenTelemetry Java Agent attached (`-javaagent`)
19+
- Export telemetry using `http/protobuf` protocol
20+
- Set faster metric export interval (5 seconds vs default 60 seconds)
21+
- Suppress MockServer logging to avoid circular telemetry
22+
23+
### Protobuf Parsing
24+
Uses OpenTelemetry protocol buffers to parse captured requests:
25+
- `ExportTraceServiceRequest` for traces/spans
26+
- `ExportMetricsServiceRequest` for metrics
27+
- Direct access to span attributes, events, and metric values
28+
29+
## What Gets Tested
30+
31+
### Traces
32+
- **HTTP spans**: Automatic instrumentation spans (e.g., `GET /rolldice`)
33+
- **Custom spans**: Manual spans created in application code (e.g., `roll-dice`, `fibonacci-calculation`)
34+
- **Span hierarchy**: Parent-child relationships between spans
35+
- **Attributes**: Custom attributes like `dice.player`, `fibonacci.n`
36+
- **Events**: Custom events like `dice-rolled`, `fibonacci-calculated`
37+
- **Error handling**: Exception recording and error status
38+
39+
### Metrics
40+
- **Custom counters**: `dice_rolls_total`, `fibonacci_calculations_total`
41+
- **Custom timers**: `dice_roll_duration_seconds`, `fibonacci_duration_seconds`
42+
- **Micrometer integration**: Metrics created via Micrometer and exported via OpenTelemetry
43+
44+
### Baggage
45+
- **Cross-cutting data**: Player names, request types
46+
- **Propagation**: Baggage values accessible across span boundaries
47+
48+
## Test Scenarios
49+
50+
### Basic Functionality
51+
```java
52+
@Test
53+
public void testDiceRollTelemetry() {
54+
// Call endpoint
55+
template.getForEntity("/rolldice", String.class);
56+
57+
// Verify spans are created
58+
await().untilAsserted(() -> {
59+
var spans = extractSpansFromRequests(requests);
60+
assertThat(spans)
61+
.extracting(Span::getName)
62+
.contains("GET /rolldice", "roll-dice", "roll-single-die");
63+
});
64+
}
65+
```
66+
67+
### Complex Scenarios
68+
- **Multiple operations**: Testing endpoints that create multiple spans
69+
- **Parameterized requests**: Verifying span attributes contain request parameters
70+
- **Error conditions**: Testing that exceptions are properly recorded
71+
- **Performance scenarios**: Large computations that generate multiple events
72+
73+
## Implementation Details
74+
75+
### Java Agent Configuration
76+
```kotlin
77+
jvmArgs = listOf(
78+
"-javaagent:${agentJarPath}",
79+
"-Dotel.metric.export.interval=5000",
80+
"-Dotel.exporter.otlp.protocol=http/protobuf",
81+
"-Dmockserver.logLevel=off"
82+
)
83+
```
84+
85+
### Request Parsing
86+
```java
87+
private List<Span> extractSpansFromRequests(HttpRequest[] requests) {
88+
return Arrays.stream(requests)
89+
.map(HttpRequest::getBody)
90+
.flatMap(body -> getExportTraceServiceRequest(body).stream())
91+
.flatMap(r -> r.getResourceSpansList().stream())
92+
.flatMap(r -> r.getScopeSpansList().stream())
93+
.flatMap(r -> r.getSpansList().stream())
94+
.collect(Collectors.toList());
95+
}
96+
```
97+
98+
## Benefits
99+
100+
### Comprehensive Validation
101+
- **End-to-end verification**: Tests actual OTLP export, not just in-memory data
102+
- **Protocol-level testing**: Uses the same protobuf format as real collectors
103+
- **Realistic conditions**: Tests with the actual Java Agent configuration
104+
105+
### Development Confidence
106+
- **Regression testing**: Catch telemetry regressions before production
107+
- **Documentation**: Tests serve as living examples of expected telemetry
108+
- **Debugging**: Easy to inspect actual telemetry data during development
109+
110+
## Running the Tests
111+
112+
```bash
113+
# Run all tests (including telemetry tests)
114+
../gradlew test
115+
116+
# Run only telemetry tests
117+
../gradlew test --tests "TelemetryTest"
118+
119+
# Run specific test method
120+
../gradlew test --tests "TelemetryTest.testDiceRollTelemetry"
121+
```
122+
123+
## Troubleshooting
124+
125+
### Common Issues
126+
127+
1. **Port conflicts**: Ensure port 4318 is available
128+
2. **Timing issues**: Use appropriate `await()` timeouts for telemetry export
129+
3. **Agent not loaded**: Verify Java Agent is properly attached in test configuration
130+
4. **Network issues**: MockServer requires localhost connectivity
131+
132+
### Debugging Tips
133+
134+
- Enable debug logging: `-Dotel.javaagent.debug=true`
135+
- Inspect raw requests: Log `request.getBodyAsString()` before parsing
136+
- Check span timing: Some spans may export in separate batches
137+
- Verify test isolation: Each test should reset MockServer expectations
138+
139+
## Further Reading
140+
141+
- [OpenTelemetry Java Agent Configuration](https://opentelemetry.io/docs/languages/java/configuration/)
142+
- [MockServer Documentation](https://www.mock-server.com/)
143+
- [OpenTelemetry Protocol Specification](https://opentelemetry.io/docs/specs/otlp/)
144+
- [Telemetry Testing Example](../telemetry-testing/README.md)

‎reference-application/build.gradle.kts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,13 @@ dependencies {
3131
// Testing
3232
testImplementation("org.springframework.boot:spring-boot-starter-test")
3333
testImplementation("io.opentelemetry:opentelemetry-sdk-testing")
34+
35+
// Telemetry testing dependencies
36+
testImplementation(enforcedPlatform("org.junit:junit-bom:5.13.4"))
37+
testImplementation("org.mockserver:mockserver-netty:5.15.0")
38+
testImplementation("org.awaitility:awaitility:4.3.0")
39+
testImplementation("io.opentelemetry.proto:opentelemetry-proto:1.8.0-alpha")
40+
testImplementation("org.assertj:assertj-core:3.27.6")
3441
}
3542

3643
val copyAgent = tasks.register<Copy>("copyAgent") {
@@ -46,6 +53,15 @@ tasks.named<BootJar>("bootJar") {
4653

4754
tasks.withType<Test> {
4855
useJUnitPlatform()
56+
57+
// Add OpenTelemetry Java Agent for telemetry testing
58+
dependsOn(copyAgent)
59+
jvmArgs = listOf(
60+
"-javaagent:${layout.buildDirectory.dir("agent").file("opentelemetry-javaagent.jar").get().asFile.absolutePath}",
61+
"-Dotel.metric.export.interval=5000",
62+
"-Dotel.exporter.otlp.protocol=http/protobuf",
63+
"-Dmockserver.logLevel=off"
64+
)
4965
}
5066

5167
task("e2eTest", Exec::class) {

0 commit comments

Comments
 (0)