Skip to content

Commit 64c521b

Browse files
zeitlingerzeitlinger
authored
Add docs and example for OTel JVM runtime metrics (#1861)
Alternative for #1854 ## Summary - Add documentation for using OpenTelemetry's runtime-telemetry module as an alternative to prometheus-metrics-instrumentation-jvm for JVM metrics following OTel semantic conventions - Add a runnable example project (examples/example-otel-jvm-runtime-metrics) demonstrating combined Prometheus + OTel runtime metrics on a single /metrics endpoint - Document RuntimeMetricsBuilder configuration options (captureGcCause(), emitExperimentalTelemetry()) - Cover standalone setup, combined setup with PrometheusMetricReader, Java 17 JFR support, and OTel-to-Prometheus metric name mapping ## Test plan - mise run build passes - mise run lint:super-linter passes - Manual: java -jar examples/example-otel-jvm-runtime-metrics/target/example-otel-jvm-runtime-metrics.jar then curl localhost:9400/metrics shows both uptime_seconds_total and jvm_* metrics --------- Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
1 parent 09bbeee commit 64c521b

File tree

7 files changed

+459
-7
lines changed

7 files changed

+459
-7
lines changed

‎.github/workflows/lint-rest.yml‎

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,13 @@ name: Lint What Super Linter Can't
33

44
on:
55
pull_request:
6-
push:
7-
branches:
8-
- main
96

10-
permissions:
11-
contents: read
7+
permissions: {}
128

139
jobs:
1410
lint:
11+
permissions:
12+
contents: read
1513
runs-on: ubuntu-24.04
1614
steps:
1715
- name: Check out
@@ -21,10 +19,16 @@ jobs:
2119
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
2220
- uses: jdx/mise-action@6d1e696aa24c1aa1bcc1adea0212707c71ab78a8 # v3.6.1
2321

22+
- name: Remap main branch URLs to PR branch for link checking
23+
env:
24+
GITHUB_HEAD_REF: ${{ github.head_ref }}
25+
PR_HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }}
26+
run: |
27+
sed -i "/^remap = \[$/a\ \"https://github.com/prometheus/client_java/blob/main/(.*) https://github.com/${PR_HEAD_REPO}/blob/${GITHUB_HEAD_REF}/\$1\"," .github/config/lychee.toml
28+
sed -i "/^remap = \[$/a\ \"https://github.com/prometheus/client_java/tree/main/(.*) https://github.com/${PR_HEAD_REPO}/tree/${GITHUB_HEAD_REF}/\$1\"," .github/config/lychee.toml
29+
2430
- name: Lint for pull requests
25-
if: github.event_name == 'pull_request'
2631
env:
2732
GITHUB_TOKEN: ${{ github.token }}
28-
GITHUB_BASE_REF: ${{ github.base_ref }}
2933
GITHUB_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
3034
run: mise run lint:rest-ci

‎docs/content/instrumentation/jvm.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,16 @@ title: JVM
33
weight: 1
44
---
55

6+
{{< hint type=note >}}
7+
8+
Looking for JVM metrics that follow OTel semantic
9+
conventions? See
10+
[OTel JVM Runtime Metrics]({{< relref "../otel/jvm-runtime-metrics.md" >}})
11+
for an alternative based on OpenTelemetry's
12+
runtime-telemetry module.
13+
14+
{{< /hint >}}
15+
616
The JVM instrumentation module provides a variety of out-of-the-box JVM and process metrics. To use
717
it, add the following dependency:
818

‎docs/content/otel/jvm-runtime-metrics.md‎

Lines changed: 238 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,238 @@
1+
---
2+
title: JVM Runtime Metrics
3+
weight: 4
4+
---
5+
6+
OpenTelemetry's
7+
[runtime-telemetry](https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/runtime-telemetry) <!-- editorconfig-checker-disable-line -->
8+
module is an alternative to
9+
[prometheus-metrics-instrumentation-jvm]({{< relref "../instrumentation/jvm.md" >}})
10+
for users who want JVM metrics following OTel semantic conventions.
11+
12+
Key advantages:
13+
14+
- Metric names follow
15+
[OTel semantic conventions](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/) <!-- editorconfig-checker-disable-line -->
16+
- Java 17+ JFR support (context switches, network I/O,
17+
lock contention, memory allocation)
18+
- Alignment with the broader OTel ecosystem
19+
20+
Since OpenTelemetry's `opentelemetry-exporter-prometheus`
21+
already depends on this library's `PrometheusRegistry`,
22+
no additional code is needed in this library — only the
23+
OTel SDK wiring shown below.
24+
25+
## Dependencies
26+
27+
{{< tabs "jvm-runtime-deps" >}}
28+
{{< tab "Gradle" >}}
29+
30+
```groovy
31+
implementation 'io.opentelemetry:opentelemetry-sdk'
32+
implementation 'io.opentelemetry:opentelemetry-exporter-prometheus'
33+
34+
// Pick ONE of the following:
35+
// Java 8+:
36+
implementation 'io.opentelemetry.instrumentation:opentelemetry-runtime-telemetry-java8'
37+
// Java 17+ (adds JFR-based metrics):
38+
// implementation 'io.opentelemetry.instrumentation:opentelemetry-runtime-telemetry-java17'
39+
```
40+
41+
{{< /tab >}}
42+
{{< tab "Maven" >}}
43+
44+
```xml
45+
<dependency>
46+
<groupId>io.opentelemetry</groupId>
47+
<artifactId>opentelemetry-sdk</artifactId>
48+
</dependency>
49+
<dependency>
50+
<groupId>io.opentelemetry</groupId>
51+
<artifactId>opentelemetry-exporter-prometheus</artifactId>
52+
</dependency>
53+
54+
<!-- Pick ONE of the following -->
55+
<!-- Java 8+ -->
56+
<dependency>
57+
<groupId>io.opentelemetry.instrumentation</groupId>
58+
<artifactId>opentelemetry-runtime-telemetry-java8</artifactId>
59+
</dependency>
60+
<!-- Java 17+ (adds JFR-based metrics) -->
61+
<!--
62+
<dependency>
63+
<groupId>io.opentelemetry.instrumentation</groupId>
64+
<artifactId>opentelemetry-runtime-telemetry-java17</artifactId>
65+
</dependency>
66+
-->
67+
```
68+
69+
{{< /tab >}}
70+
{{< /tabs >}}
71+
72+
## Standalone Setup
73+
74+
If you **only** want OTel runtime metrics exposed as
75+
Prometheus, without any Prometheus Java client metrics:
76+
77+
```java
78+
import io.opentelemetry.exporter.prometheus.PrometheusHttpServer;
79+
import io.opentelemetry.instrumentation.runtimemetrics.java8.RuntimeMetrics;
80+
import io.opentelemetry.sdk.OpenTelemetrySdk;
81+
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
82+
83+
PrometheusHttpServer prometheusServer =
84+
PrometheusHttpServer.builder()
85+
.setPort(9464)
86+
.build();
87+
88+
OpenTelemetrySdk openTelemetry =
89+
OpenTelemetrySdk.builder()
90+
.setMeterProvider(
91+
SdkMeterProvider.builder()
92+
.registerMetricReader(prometheusServer)
93+
.build())
94+
.build();
95+
96+
RuntimeMetrics runtimeMetrics =
97+
RuntimeMetrics.builder(openTelemetry).build();
98+
99+
// Close on shutdown to stop metric collection and server
100+
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
101+
runtimeMetrics.close();
102+
prometheusServer.close();
103+
}));
104+
105+
// Scrape at http://localhost:9464/metrics
106+
```
107+
108+
## Combined with Prometheus Java Client Metrics
109+
110+
If you already have Prometheus Java client metrics and want to
111+
add OTel runtime metrics to the **same** `/metrics`
112+
endpoint, use `PrometheusMetricReader` to bridge OTel
113+
metrics into a `PrometheusRegistry`:
114+
115+
```java
116+
import io.prometheus.metrics.core.metrics.Counter;
117+
import io.prometheus.metrics.exporter.httpserver.HTTPServer;
118+
import io.prometheus.metrics.model.registry.PrometheusRegistry;
119+
import io.opentelemetry.exporter.prometheus.PrometheusMetricReader;
120+
import io.opentelemetry.instrumentation.runtimemetrics.java8.RuntimeMetrics;
121+
import io.opentelemetry.sdk.OpenTelemetrySdk;
122+
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
123+
124+
PrometheusRegistry registry =
125+
new PrometheusRegistry();
126+
127+
// Register Prometheus metrics as usual
128+
Counter myCounter = Counter.builder()
129+
.name("my_requests_total")
130+
.register(registry);
131+
132+
// Bridge OTel metrics into the same registry
133+
PrometheusMetricReader reader =
134+
PrometheusMetricReader.create();
135+
registry.register(reader);
136+
137+
OpenTelemetrySdk openTelemetry =
138+
OpenTelemetrySdk.builder()
139+
.setMeterProvider(
140+
SdkMeterProvider.builder()
141+
.registerMetricReader(reader)
142+
.build())
143+
.build();
144+
145+
RuntimeMetrics runtimeMetrics =
146+
RuntimeMetrics.builder(openTelemetry).build();
147+
Runtime.getRuntime()
148+
.addShutdownHook(new Thread(runtimeMetrics::close));
149+
150+
// Expose everything on one endpoint
151+
HTTPServer.builder()
152+
.port(9400)
153+
.registry(registry)
154+
.buildAndStart();
155+
```
156+
157+
The [examples/example-otel-jvm-runtime-metrics](https://github.com/prometheus/client_java/tree/main/examples/example-otel-jvm-runtime-metrics) <!-- editorconfig-checker-disable-line -->
158+
directory has a complete runnable example.
159+
160+
## Configuration
161+
162+
The `RuntimeMetricsBuilder` supports two configuration
163+
options:
164+
165+
### `captureGcCause()`
166+
167+
Adds a `jvm.gc.cause` attribute to the `jvm.gc.duration`
168+
metric, indicating why the garbage collection occurred
169+
(e.g. `G1 Evacuation Pause`, `System.gc()`):
170+
171+
```java
172+
RuntimeMetrics.builder(openTelemetry)
173+
.captureGcCause()
174+
.build();
175+
```
176+
177+
### `emitExperimentalTelemetry()`
178+
179+
Enables additional experimental metrics beyond the stable
180+
set. These are not yet part of the OTel semantic conventions
181+
and may change in future releases:
182+
183+
- Buffer pool metrics (direct and mapped byte buffers)
184+
- Extended CPU metrics
185+
- Extended memory pool metrics
186+
- File descriptor metrics
187+
188+
```java
189+
RuntimeMetrics.builder(openTelemetry)
190+
.emitExperimentalTelemetry()
191+
.build();
192+
```
193+
194+
Both options can be combined:
195+
196+
```java
197+
RuntimeMetrics.builder(openTelemetry)
198+
.captureGcCause()
199+
.emitExperimentalTelemetry()
200+
.build();
201+
```
202+
203+
Selective per-metric registration is not supported by the
204+
runtime-telemetry API — it is all-or-nothing with these
205+
two toggles.
206+
207+
## Java 17 JFR Support
208+
209+
The `opentelemetry-runtime-telemetry-java17` variant adds
210+
JFR-based metrics. You can selectively enable features:
211+
212+
```java
213+
import io.opentelemetry.instrumentation.runtimemetrics.java17.JfrFeature;
214+
import io.opentelemetry.instrumentation.runtimemetrics.java17.RuntimeMetrics;
215+
216+
RuntimeMetrics.builder(openTelemetry)
217+
.enableFeature(JfrFeature.BUFFER_METRICS)
218+
.enableFeature(JfrFeature.NETWORK_IO_METRICS)
219+
.enableFeature(JfrFeature.LOCK_METRICS)
220+
.enableFeature(JfrFeature.CONTEXT_SWITCH_METRICS)
221+
.build();
222+
```
223+
224+
## Metric Names
225+
226+
OTel metric names are converted to Prometheus format by
227+
the exporter. Examples:
228+
229+
| OTel name | Prometheus name |
230+
| ---------------------------- | ---------------------------------- |
231+
| `jvm.memory.used` | `jvm_memory_used_bytes` |
232+
| `jvm.gc.duration` | `jvm_gc_duration_seconds` |
233+
| `jvm.thread.count` | `jvm_thread_count` |
234+
| `jvm.class.loaded` | `jvm_class_loaded` |
235+
| `jvm.cpu.recent_utilization` | `jvm_cpu_recent_utilization_ratio` |
236+
237+
See [Names]({{< relref "names.md" >}}) for full details on
238+
how OTel names map to Prometheus names.

‎examples/example-otel-jvm-runtime-metrics/README.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# OTel JVM Runtime Metrics with Prometheus HTTPServer
2+
3+
## Build
4+
5+
This example is built as part of the `client_java` project.
6+
7+
```shell
8+
./mvnw package
9+
```
10+
11+
## Run
12+
13+
The build creates a JAR file with the example application in
14+
`./examples/example-otel-jvm-runtime-metrics/target/`.
15+
16+
```shell
17+
java -jar ./examples/example-otel-jvm-runtime-metrics/target/example-otel-jvm-runtime-metrics.jar
18+
```
19+
20+
## Manually Testing the Metrics Endpoint
21+
22+
Accessing
23+
[http://localhost:9400/metrics](http://localhost:9400/metrics)
24+
with a Web browser should yield both a Prometheus counter metric
25+
and OTel JVM runtime metrics on the same endpoint.
26+
27+
Prometheus counter:
28+
29+
```text
30+
# HELP uptime_seconds_total total number of seconds since this application was started
31+
# TYPE uptime_seconds_total counter
32+
uptime_seconds_total 42.0
33+
```
34+
35+
OTel JVM runtime metrics (excerpt):
36+
37+
```text
38+
# HELP jvm_memory_used_bytes Measure of memory used.
39+
# TYPE jvm_memory_used_bytes gauge
40+
jvm_memory_used_bytes{jvm_memory_pool_name="G1 Eden Space",jvm_memory_type="heap"} 4194304.0
41+
```

0 commit comments

Comments
 (0)