You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(profiling): Add original_payload support to OTLP profiles converter
Implement support for OTLP profiles original_payload and original_payload_format fields (fields 9 and 10) to include source JFR recording(s) in OTLP output for debugging and compliance verification.
Key features:
- Zero-copy streaming architecture using SequenceInputStream
- Automatic uber-JFR concatenation for multiple recordings
- Disabled by default per OTLP spec recommendation (size considerations)
- Fluent API: setIncludeOriginalPayload(boolean)
Implementation details:
- Enhanced ProtobufEncoder with streaming writeBytesField(InputStream, long) method
- Single file optimization: direct FileInputStream
- Multiple files: SequenceInputStream chains files with zero memory overhead
- Streams data in 8KB chunks directly into protobuf output
Test coverage:
- Default behavior verification (payload disabled)
- Single file with payload enabled
- Multiple files creating uber-JFR concatenation
- Setting persistence across converter reuse
Documentation:
- Added Phase 6 to ARCHITECTURE.md with usage examples, design decisions, and performance characteristics
- Centralized jafar-parser dependency version in gradle/libs.versions.toml
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
Copy file name to clipboardExpand all lines: dd-java-agent/agent-profiling/profiling-otel/doc/ARCHITECTURE.md
+187-1Lines changed: 187 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -508,7 +508,193 @@ Potential improvements if cache effectiveness needs to be increased:
508
508
509
509
This optimization targets the real bottleneck (redundant frame processing) rather than micro-optimizing already-efficient dictionary operations, resulting in measurable improvements for production workloads with realistic stack duplication patterns.
Implement support for OTLP profiles `original_payload` and `original_payload_format` fields (fields 9 and 10) to include the source JFR recording(s) in OTLP output for debugging and compliance purposes.
516
+
517
+
#### OTLP Specification Context
518
+
519
+
Per [OTLP profiles.proto v1development](https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/profiles/v1development/profiles.proto#L337):
520
+
521
+
-`original_payload_format` (field 9): String indicating the format of the original recording (e.g., "jfr", "pprof")
522
+
-`original_payload` (field 10): Raw bytes of the original profiling data
523
+
524
+
**Note**: The OTLP spec recommends this feature be **disabled by default** due to payload size considerations. It is intended for debugging OTLP content and compliance verification, not routine production use.
525
+
526
+
#### Implementation Details
527
+
528
+
**API Design:**
529
+
530
+
```java
531
+
// Disabled by default per OTLP spec recommendation
532
+
converter.setIncludeOriginalPayload(true)
533
+
.addFile(jfrFile, start, end)
534
+
.convert();
535
+
```
536
+
537
+
**Key Features:**
538
+
539
+
1.**Zero-Copy Streaming** - JFR recordings are streamed directly into protobuf output without memory allocation:
540
+
- Single file: Direct `FileInputStream`
541
+
- Multiple files: `SequenceInputStream` chains files together
542
+
- Protobuf encoder streams data in 8KB chunks
543
+
544
+
2.**Uber-JFR Concatenation** - Multiple JFR recordings are automatically concatenated:
545
+
- JFR format supports concatenation natively (multiple chunks in sequence)
546
+
-`SequenceInputStream` chains file streams using `Enumeration<InputStream>` wrapper
547
+
- Protobuf length-delimited encoding preserves total byte count
548
+
549
+
3.**Enhanced ProtobufEncoder** - New streaming method for large payloads:
550
+
```java
551
+
publicvoid writeBytesField(int fieldNumber, InputStream inputStream, long length)
- Tests setting persistence across multiple `convert()` calls
642
+
- Verifies fluent API behavior and converter reuse
643
+
644
+
**Size Validation Strategy**: Since we cannot easily parse protobuf bytes in tests, we validate by comparing output size. When `original_payload` is included, the total output size must be >= source JFR file size(s), as it contains both OTLP profile data AND the raw JFR bytes.
645
+
646
+
#### Performance Characteristics
647
+
648
+
**Memory Efficiency:**
649
+
-**Streaming I/O**: No memory allocation for JFR content
650
+
- Single-file optimization: Direct `FileInputStream` (no wrapper overhead)
0 commit comments