Skip to content

Commit 99f80e4

Browse files
SylvainJugeSylvainJuge
authored
jmx reuse instrumentation metrics (#1782)
1 parent 2b31033 commit 99f80e4

File tree

10 files changed

+262
-115
lines changed

10 files changed

+262
-115
lines changed

‎jmx-scraper/README.md

Lines changed: 29 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ For example the `otel.jmx.service.url` option can be set with the `OTEL_JMX_SERV
4040
|--------------------------------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
4141
| `otel.jmx.service.url` | - | mandatory JMX URL to connect to the remote JVM |
4242
| `otel.jmx.target.system` | - | comma-separated list of systems to monitor, mandatory unless `otel.jmx.config` is set |
43+
| `otel.jmx.target.source` | `auto` | source of metrics definitions to use for `otel.jmx.target.system`, supported values are `auto`, `instrumentation` and `legacy` |
4344
| `otel.jmx.config` | empty | comma-separated list of paths to custom YAML metrics definition, mandatory when `otel.jmx.target.system` is not set |
4445
| `otel.jmx.username` | - | user name for JMX connection, mandatory when JMX authentication is set on target JVM with`com.sun.management.jmxremote.authenticate=true` |
4546
| `otel.jmx.password` | - | password for JMX connection, mandatory when JMX authentication is set on target JVM with `com.sun.management.jmxremote.authenticate=true` |
@@ -55,22 +56,34 @@ When both `otel.jmx.target.system` and `otel.jmx.config` configuration options a
5556

5657
If there is a need to override existing ready-to-use metrics or to keep control on the metrics definitions, using a custom YAML definition with `otel.jmx.config` is the recommended option.
5758

58-
Supported values for `otel.jmx.target.system`:
59-
60-
| `otel.jmx.target.system` | description |
61-
|--------------------------|-----------------------|
62-
| `activemq` | Apache ActiveMQ |
63-
| `cassandra` | Apache Cassandra |
64-
| `hbase` | Apache HBase |
65-
| `hadoop` | Apache Hadoop |
66-
| `jetty` | Eclipse Jetty |
67-
| `jvm` | JVM runtime metrics |
68-
| `kafka` | Apache Kafka |
69-
| `kafka-consumer` | Apache Kafka consumer |
70-
| `kafka-producer` | Apache Kafka producer |
71-
| `solr` | Apache Solr |
72-
| `tomcat` | Apache Tomcat |
73-
| `wildfly` | Wildfly |
59+
Supported values for `otel.jmx.target.system` and support for `otel.jmx.target.source` and links to the metrics definitions:
60+
61+
| `otel.jmx.target.system` | description | `legacy` | `instrumentation` |
62+
|--------------------------|-----------------------|-----------------------------------------------------------------|-------------------|
63+
| `activemq` | Apache ActiveMQ | [`activemq.yaml`](src/main/resources/activemq.yaml) | |
64+
| `cassandra` | Apache Cassandra | [`cassandra.yaml`](src/main/resources/cassandra.yaml) | |
65+
| `hbase` | Apache HBase | [`hbase.yaml`](src/main/resources/hbase.yaml) | |
66+
| `hadoop` | Apache Hadoop | [`hadoop.yaml`](src/main/resources/hadoop.yaml) | |
67+
| `jetty` | Eclipse Jetty | [`jetty.yaml`](src/main/resources/jetty.yaml) | |
68+
| `jvm` | JVM runtime metrics | [`jvm.yaml`](src/main/resources/jvm.yaml) | |
69+
| `kafka` | Apache Kafka | [`kafka.yaml`](src/main/resources/kafka.yaml) | |
70+
| `kafka-consumer` | Apache Kafka consumer | [`kafka-consumer.yaml`](src/main/resources/kafka-consumer.yaml) | |
71+
| `kafka-producer` | Apache Kafka producer | [`kafka-producer.yaml`](src/main/resources/kafka-producer.yaml) | |
72+
| `solr` | Apache Solr | [`solr.yaml`](src/main/resources/solr.yaml) | |
73+
| `tomcat` | Apache Tomcat | [`tomcat.yaml`](src/main/resources/tomcat.yaml) | |
74+
| `wildfly` | Wildfly | [`wildfly.yaml`](src/main/resources/wildfly.yaml) | |
75+
76+
The source of metrics definitions is controlled by `otel.jmx.target.source`:
77+
78+
- `auto` (default) : metrics definitions from `instrumentation` with fallback on `legacy` when not available.
79+
- `legacy` : metrics definitions embedded in jmx-scraper, almost equivalent to [JMX Gatherer](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/jmx-metrics).
80+
- `instrumentation` : metrics definitions embedded in [instrumentation/jmx-metrics](https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/jmx-metrics/library) library
81+
82+
Setting the value of `otel.jmx.target.source` allows to fit the following use-cases:
83+
84+
- `auto` will ensure that the latest metrics definitions in instrumentation (reference) is being used when available with a fallback on `legacy` otherwise. Metrics definitions will thus be updated whenever the dependency on instrumentation is updated.
85+
- `legacy` allows to keep using definitions that are very close to JMX Gatherer, this is the recommended option if preserving compatibility is required. Those definitions are in maintenance and are unlikely to evolve over time.
86+
- `instrumentation` forces using metrics definitions from instrumentation, hence only the reference. Metrics definitions and supported values of `otel.jmx.target.system` will be updated whenever the dependency on instrumentation is updated.
7487

7588
The following SDK configuration options are also relevant
7689

‎jmx-scraper/src/integrationTest/java/io/opentelemetry/contrib/jmxscraper/JmxScraperContainer.java

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ public class JmxScraperContainer extends GenericContainer<JmxScraperContainer> {
2626

2727
private final String endpoint;
2828
private final Set<String> targetSystems;
29+
private String targetSystemSource;
2930
private String serviceUrl;
3031
private final Set<String> customYamlFiles;
3132
private String user;
@@ -46,6 +47,7 @@ public JmxScraperContainer(String otlpEndpoint, String baseImage) {
4647

4748
this.endpoint = otlpEndpoint;
4849
this.targetSystems = new HashSet<>();
50+
this.targetSystemSource = "auto";
4951
this.customYamlFiles = new HashSet<>();
5052
this.extraJars = new ArrayList<>();
5153
}
@@ -62,6 +64,18 @@ public JmxScraperContainer withTargetSystem(String targetSystem) {
6264
return this;
6365
}
6466

67+
/**
68+
* Sets the target system source
69+
*
70+
* @param source target system source, valid values are "auto", "instrumentation" and "legacy"
71+
* @return this
72+
*/
73+
@CanIgnoreReturnValue
74+
public JmxScraperContainer withTargetSystemSource(String source) {
75+
this.targetSystemSource = source;
76+
return this;
77+
}
78+
6579
/**
6680
* Set connection to a standard JMX service URL
6781
*
@@ -192,6 +206,7 @@ public void start() {
192206

193207
if (!targetSystems.isEmpty()) {
194208
arguments.add("-Dotel.jmx.target.system=" + String.join(",", targetSystems));
209+
arguments.add("-Dotel.jmx.target.source=" + targetSystemSource);
195210
}
196211

197212
if (serviceUrl == null) {

‎jmx-scraper/src/integrationTest/java/io/opentelemetry/contrib/jmxscraper/target_systems/JvmIntegrationTest.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,12 @@ protected JmxScraperContainer customizeScraperContainer(
3232
JmxScraperContainer scraper, GenericContainer<?> target, Path tempDir) {
3333
return scraper
3434
.withTargetSystem("jvm")
35+
// TODO when JVM metrics will be added to instrumentation, the default "auto" source
36+
// means that the definitions in instrumentation will be used, and thus this test will fail
37+
// due to metrics differences, adding an explicit "legacy" source is required to continue
38+
// testing the JVM metrics defined in this project.
39+
// https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/13392
40+
// .withTargetSystem("legacy")
3541
// also testing custom yaml
3642
.withCustomYaml("custom-metrics.yaml");
3743
}

‎jmx-scraper/src/main/java/io/opentelemetry/contrib/jmxscraper/JmxScraper.java

Lines changed: 5 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -228,28 +228,18 @@ private void start() throws IOException {
228228
private static MetricConfiguration getMetricConfig(JmxScraperConfig scraperConfig) {
229229
MetricConfiguration config = new MetricConfiguration();
230230
for (String system : scraperConfig.getTargetSystems()) {
231-
addRulesForSystem(system, config);
231+
try (InputStream yaml = scraperConfig.getTargetSystemYaml(system)) {
232+
RuleParser.get().addMetricDefsTo(config, yaml, system);
233+
} catch (IOException e) {
234+
throw new IllegalStateException("Error while loading rules for system " + system, e);
235+
}
232236
}
233237
for (String file : scraperConfig.getJmxConfig()) {
234238
addRulesFromFile(file, config);
235239
}
236240
return config;
237241
}
238242

239-
private static void addRulesForSystem(String system, MetricConfiguration conf) {
240-
String yamlResource = system + ".yaml";
241-
try (InputStream inputStream =
242-
JmxScraper.class.getClassLoader().getResourceAsStream(yamlResource)) {
243-
if (inputStream != null) {
244-
RuleParser.get().addMetricDefsTo(conf, inputStream, system);
245-
} else {
246-
throw new IllegalArgumentException("No support for system " + system);
247-
}
248-
} catch (Exception e) {
249-
throw new IllegalStateException("Error while loading rules for system " + system, e);
250-
}
251-
}
252-
253243
private static void addRulesFromFile(String file, MetricConfiguration conf) {
254244
Path path = Paths.get(file);
255245
if (!Files.isReadable(path)) {

‎jmx-scraper/src/main/java/io/opentelemetry/contrib/jmxscraper/config/JmxScraperConfig.java

Lines changed: 77 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,13 @@
77

88
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
99
import io.opentelemetry.sdk.autoconfigure.spi.ConfigurationException;
10+
import java.io.InputStream;
1011
import java.time.Duration;
1112
import java.util.ArrayList;
12-
import java.util.Arrays;
1313
import java.util.Collections;
1414
import java.util.HashSet;
1515
import java.util.List;
16+
import java.util.Locale;
1617
import java.util.Set;
1718
import java.util.logging.Logger;
1819
import javax.annotation.Nullable;
@@ -37,37 +38,23 @@ public class JmxScraperConfig {
3738
static final String JMX_CONFIG_LEGACY = "otel.jmx.custom.scraping.config";
3839

3940
static final String JMX_TARGET_SYSTEM = "otel.jmx.target.system";
41+
static final String JMX_TARGET_SOURCE = "otel.jmx.target.source";
4042

4143
static final String JMX_USERNAME = "otel.jmx.username";
4244
static final String JMX_PASSWORD = "otel.jmx.password";
4345

44-
// TODO: document those when they will be supported
4546
static final String JMX_REGISTRY_SSL = "otel.jmx.remote.registry.ssl";
4647
static final String JMX_REMOTE_PROFILE = "otel.jmx.remote.profile";
4748
static final String JMX_REALM = "otel.jmx.realm";
4849

49-
private static final List<String> AVAILABLE_TARGET_SYSTEMS =
50-
Collections.unmodifiableList(
51-
Arrays.asList(
52-
"activemq",
53-
"cassandra",
54-
"hbase",
55-
"hadoop",
56-
"jetty",
57-
"jvm",
58-
"kafka",
59-
"kafka-consumer",
60-
"kafka-producer",
61-
"solr",
62-
"tomcat",
63-
"wildfly"));
64-
6550
private String serviceUrl = "";
6651

6752
private List<String> jmxConfig = Collections.emptyList();
6853

6954
private Set<String> targetSystems = Collections.emptySet();
7055

56+
private TargetSystemSource targetSystemSource = TargetSystemSource.AUTO;
57+
7158
private Duration samplingInterval = Duration.ofMinutes(1);
7259

7360
@Nullable private String username;
@@ -79,6 +66,20 @@ public class JmxScraperConfig {
7966
@Nullable private String remoteProfile;
8067
private boolean registrySsl;
8168

69+
public enum TargetSystemSource {
70+
AUTO,
71+
INSTRUMENTATION,
72+
LEGACY;
73+
74+
static TargetSystemSource fromString(String source) {
75+
try {
76+
return TargetSystemSource.valueOf(source.toUpperCase(Locale.ROOT));
77+
} catch (IllegalArgumentException e) {
78+
throw new IllegalArgumentException("Invalid target system source: " + source, e);
79+
}
80+
}
81+
}
82+
8283
private JmxScraperConfig() {}
8384

8485
public String getServiceUrl() {
@@ -93,6 +94,57 @@ public Set<String> getTargetSystems() {
9394
return targetSystems;
9495
}
9596

97+
/**
98+
* Resolves the target system yaml from configuration
99+
*
100+
* @param system target system
101+
* @return input stream on target system yaml definitions
102+
* @throws ConfigurationException when no yaml for system is available
103+
*/
104+
public InputStream getTargetSystemYaml(String system) {
105+
InputStream yaml;
106+
switch (targetSystemSource) {
107+
case LEGACY:
108+
yaml = getTargetSystemYaml(system, TargetSystemSource.LEGACY);
109+
break;
110+
case INSTRUMENTATION:
111+
yaml = getTargetSystemYaml(system, TargetSystemSource.INSTRUMENTATION);
112+
break;
113+
case AUTO:
114+
yaml = getTargetSystemYaml(system, TargetSystemSource.INSTRUMENTATION);
115+
if (yaml == null) {
116+
yaml = getTargetSystemYaml(system, TargetSystemSource.LEGACY);
117+
}
118+
break;
119+
default:
120+
throw new IllegalStateException("unsupported target system source: " + targetSystemSource);
121+
}
122+
123+
if (yaml == null) {
124+
throw new ConfigurationException(
125+
"unsupported target system: '" + system + "', source: " + targetSystemSource);
126+
}
127+
return yaml;
128+
}
129+
130+
@Nullable
131+
private static InputStream getTargetSystemYaml(String system, TargetSystemSource source) {
132+
String path;
133+
switch (source) {
134+
case LEGACY:
135+
path = String.format("%s.yaml", system);
136+
break;
137+
case INSTRUMENTATION:
138+
path = String.format("jmx/rules/%s.yaml", system);
139+
break;
140+
case AUTO:
141+
default:
142+
throw new IllegalArgumentException("invalid source" + source);
143+
}
144+
145+
return JmxScraperConfig.class.getClassLoader().getResourceAsStream(path);
146+
}
147+
96148
public Duration getSamplingInterval() {
97149
return samplingInterval;
98150
}
@@ -164,12 +216,6 @@ public static JmxScraperConfig fromConfig(ConfigProperties config) {
164216
throw new ConfigurationException(
165217
"at least one of '" + JMX_TARGET_SYSTEM + "' or '" + JMX_CONFIG + "' must be set");
166218
}
167-
targetSystem.forEach(
168-
s -> {
169-
if (!AVAILABLE_TARGET_SYSTEMS.contains(s)) {
170-
throw new ConfigurationException("unsupported target system: '" + s + "'");
171-
}
172-
});
173219

174220
scraperConfig.jmxConfig = Collections.unmodifiableList(jmxConfig);
175221
scraperConfig.targetSystems = Collections.unmodifiableSet(new HashSet<>(targetSystem));
@@ -180,6 +226,13 @@ public static JmxScraperConfig fromConfig(ConfigProperties config) {
180226
scraperConfig.realm = config.getString("otel.jmx.realm");
181227
scraperConfig.registrySsl = config.getBoolean("otel.jmx.remote.registry.ssl", false);
182228

229+
// checks target system is supported by resolving the yaml resource, throws exception on
230+
// missing/error
231+
scraperConfig.targetSystems.forEach(scraperConfig::getTargetSystemYaml);
232+
233+
String source = config.getString(JMX_TARGET_SOURCE, TargetSystemSource.AUTO.name());
234+
scraperConfig.targetSystemSource = TargetSystemSource.fromString(source);
235+
183236
return scraperConfig;
184237
}
185238
}

0 commit comments

Comments
 (0)