Skip to content

Commit ed61c40

Browse files
jbachorikclaude
jbachorik
and
claude
committed
feat(profiling): Add CLI tool for converting JFR to OTLP format
Add command-line interface for testing and validating JFR to OTLP conversions with real profiling data. Features: - Convert single or multiple JFR files to OTLP protobuf or JSON - Include original JFR payload for validation (optional) - Merge multiple recordings into single output - Detailed conversion statistics Usage: ./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \ -Pargs="recording.jfr output.pb" ./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \ -Pargs="--json recording.jfr output.json" See doc/CLI.md for complete documentation and examples. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent 9efd530 commit ed61c40

File tree

3 files changed

+440
-0
lines changed

3 files changed

+440
-0
lines changed

‎dd-java-agent/agent-profiling/profiling-otel/build.gradle.kts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,18 @@ tasks.named<JavaCompile>("compileJmhJava") {
7575
)
7676
}
7777

78+
// CLI task for converting JFR files
79+
// Usage: ./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr --args="input.jfr output.pb"
80+
// Usage: ./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr --args="--json input.jfr output.json"
81+
tasks.register<JavaExec>("convertJfr") {
82+
group = "application"
83+
description = "Convert JFR recording to OTLP profiles format"
84+
classpath = sourceSets["main"].runtimeClasspath
85+
mainClass.set("com.datadog.profiling.otel.JfrToOtlpConverterCLI")
86+
87+
// Uses Gradle's built-in --args parameter which properly handles spaces in paths
88+
}
89+
7890
dependencies {
7991
implementation(libs.jafar.parser)
8092
implementation(project(":internal-api"))

‎dd-java-agent/agent-profiling/profiling-otel/doc/CLI.md‎

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
# JFR to OTLP Converter CLI
2+
3+
Command-line tool for converting JFR recordings to OTLP profiles format for testing and validation.
4+
5+
## Quick Start
6+
7+
Convert a JFR file to OTLP protobuf format:
8+
9+
```bash
10+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr --args="recording.jfr output.pb"
11+
```
12+
13+
Convert to JSON for human inspection:
14+
15+
```bash
16+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr --args="--json recording.jfr output.json"
17+
```
18+
19+
## Usage
20+
21+
```bash
22+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr --args="[options] input.jfr [input2.jfr ...] output"
23+
```
24+
25+
### Options
26+
27+
- `--json` - Output JSON format instead of protobuf (useful for inspection)
28+
- `--include-payload` - Include original JFR payload in output (increases size significantly)
29+
- `--help` - Show help message
30+
31+
### Examples
32+
33+
#### Basic Conversion
34+
35+
Convert single JFR to protobuf:
36+
37+
```bash
38+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
39+
--args="recording.jfr output.pb"
40+
```
41+
42+
#### JSON Output for Inspection
43+
44+
Output JSON format to examine the structure:
45+
46+
```bash
47+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
48+
--args="--json recording.jfr output.json"
49+
50+
# Inspect with jq
51+
cat output.json | jq '.dictionary.string_table | length'
52+
cat output.json | jq '.resource_profiles[0].scope_profiles[0].profiles[] | .sample_type'
53+
```
54+
55+
#### Merge Multiple Recordings
56+
57+
Combine multiple JFR files into a single OTLP output:
58+
59+
```bash
60+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
61+
--args="recording1.jfr recording2.jfr recording3.jfr merged.pb"
62+
```
63+
64+
This is useful for:
65+
- Merging recordings from different time periods
66+
- Combining CPU and allocation profiles
67+
- Testing dictionary deduplication across files
68+
69+
#### Include Original Payload
70+
71+
Include the original JFR data in the OTLP output:
72+
73+
```bash
74+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
75+
--args="--include-payload recording.jfr output.pb"
76+
```
77+
78+
**Note**: This significantly increases output size (typically 2-3x) as it embeds the entire JFR file(s) in the `original_payload` field.
79+
80+
## Output Analysis
81+
82+
The CLI prints conversion statistics:
83+
84+
```
85+
Converting 1 JFR file(s) to OTLP format...
86+
Adding: /path/to/recording.jfr
87+
Conversion complete!
88+
Output: /path/to/output.pb
89+
Format: PROTO
90+
Size: 45.2 KB
91+
Time: 127 ms
92+
```
93+
94+
With `--include-payload`:
95+
96+
```
97+
Converting 1 JFR file(s) to OTLP format...
98+
Adding: /path/to/recording.jfr
99+
Conversion complete!
100+
Output: /path/to/output.pb
101+
Format: PROTO
102+
Size: 125.7 KB
103+
Time: 134 ms
104+
Input size: 89.3 KB
105+
Compression: 140.8%
106+
```
107+
108+
**Note**: When including the original payload, the output may be *larger* than the input due to protobuf overhead. The primary benefit of original_payload is preserving the raw data for alternative processing, not compression.
109+
110+
## Inspecting JSON Output
111+
112+
The JSON output contains the complete OTLP structure:
113+
114+
```json
115+
{
116+
"resource_profiles": [{
117+
"scope_profiles": [{
118+
"profiles": [{
119+
"sample_type": { "type_strindex": 1, "unit_strindex": 2 },
120+
"samples": [
121+
{ "stack_index": 1, "link_index": 2, "values": [1], "timestamps_unix_nano": [1234567890000000] }
122+
],
123+
"time_unix_nano": 1234567800000000000,
124+
"duration_nano": 60000000000,
125+
"profile_id": "a1b2c3d4..."
126+
}]
127+
}]
128+
}],
129+
"dictionary": {
130+
"location_table": [...],
131+
"function_table": [...],
132+
"link_table": [...],
133+
"string_table": ["", "cpu", "samples", "com.example.Class", ...],
134+
"stack_table": [...]
135+
}
136+
}
137+
```
138+
139+
Key fields to inspect:
140+
141+
```bash
142+
# Count samples by profile type
143+
cat output.json | jq '.resource_profiles[0].scope_profiles[0].profiles[] | "\(.sample_type.type_strindex): \(.samples | length)"'
144+
145+
# Show dictionary sizes
146+
cat output.json | jq '.dictionary | {strings: (.string_table | length), functions: (.function_table | length), locations: (.location_table | length), stacks: (.stack_table | length)}'
147+
148+
# Show first 10 stack frames
149+
cat output.json | jq '.dictionary.string_table[1:10]'
150+
151+
# Find deepest stack
152+
cat output.json | jq '.dictionary.stack_table | max_by(.location_indices | length)'
153+
```
154+
155+
## Testing Real JFR Files
156+
157+
To test with production JFR recordings:
158+
159+
1. **Generate test recording**:
160+
```bash
161+
# Start profiling
162+
jcmd <pid> JFR.start name=test duration=60s filename=test.jfr
163+
164+
# Wait for recording
165+
sleep 60
166+
167+
# Convert to OTLP
168+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
169+
--args="test.jfr output.pb"
170+
```
171+
172+
2. **Use existing recording**:
173+
```bash
174+
# Find JFR files
175+
find /tmp -name "*.jfr" 2>/dev/null
176+
177+
# Convert the most recent
178+
latest=$(ls -t /tmp/*.jfr | head -1)
179+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
180+
--args="--json $latest output.json"
181+
```
182+
183+
3. **Compare formats**:
184+
```bash
185+
# Original JFR size
186+
ls -lh recording.jfr
187+
188+
# OTLP protobuf size
189+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
190+
--args="recording.jfr output.pb"
191+
ls -lh output.pb
192+
193+
# OTLP JSON size (larger due to text encoding)
194+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
195+
--args="--json recording.jfr output.json"
196+
ls -lh output.json
197+
```
198+
199+
## Performance Testing
200+
201+
For performance benchmarks, use the JMH benchmarks instead:
202+
203+
```bash
204+
# Run end-to-end conversion benchmark
205+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:jmh \
206+
-PjmhIncludes="JfrToOtlpConverterBenchmark"
207+
```
208+
209+
See [BENCHMARKS.md](BENCHMARKS.md) for details.
210+
211+
## Troubleshooting
212+
213+
### "Input file not found"
214+
215+
Ensure the JFR file path is correct and accessible:
216+
217+
```bash
218+
ls -l recording.jfr
219+
```
220+
221+
### "Error parsing JFR file"
222+
223+
The JFR file may be corrupted or incomplete. Validate with:
224+
225+
```bash
226+
jfr print --events recording.jfr
227+
```
228+
229+
### Gradle task not found
230+
231+
Ensure you're using the full task path:
232+
233+
```bash
234+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr --args="..."
235+
```
236+
237+
### Out of memory
238+
239+
For very large JFR files, increase heap:
240+
241+
```bash
242+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:convertJfr \
243+
--args="large.jfr output.pb" \
244+
--max-workers=1 \
245+
-Dorg.gradle.jvmargs="-Xmx2g"
246+
```
247+
248+
## Direct Java Execution
249+
250+
For scripting or CI/CD, you can run the CLI directly after building:
251+
252+
```bash
253+
# Build the project
254+
./gradlew :dd-java-agent:agent-profiling:profiling-otel:jar
255+
256+
# Run directly with java
257+
java -cp "dd-java-agent/agent-profiling/profiling-otel/build/libs/*:$(find . -name 'jafar-parser*.jar'):$(find internal-api -name '*.jar'):$(find components/json -name '*.jar')" \
258+
com.datadog.profiling.otel.JfrToOtlpConverterCLI \
259+
recording.jfr output.pb
260+
```
261+
262+
**Note**: Managing the classpath manually is complex. The Gradle task is recommended.
263+
264+
## See Also
265+
266+
- [ARCHITECTURE.md](ARCHITECTURE.md) - Converter design and implementation details
267+
- [BENCHMARKS.md](BENCHMARKS.md) - Performance benchmarks and profiling
268+
- [../README.md](../README.md) - Module overview

0 commit comments

Comments
 (0)