jmx scraper with sdk autoconfig

jmx-scraper/README.md

@@ -21,20 +21,70 @@ Minimal configuration required
 Configuration can be provided through:
 
 - command line arguments:
-  `java -jar scraper.jar --config otel.jmx.service.url=service:jmx:rmi:///jndi/rmi://tomcat:9010/jmxrmi otel.jmx.target.system=tomcat`.
+  `java -jar scraper.jar -config otel.jmx.service.url=service:jmx:rmi:///jndi/rmi://tomcat:9010/jmxrmi otel.jmx.target.system=tomcat`.
 - command line arguments JVM system properties:
   `java -Dotel.jmx.service.url=service:jmx:rmi:///jndi/rmi://tomcat:9010/jmxrmi -Dotel.jmx.target.system=tomcat -jar scraper.jar`.
 - java properties file: `java -jar scraper.jar -config config.properties`.
 - stdin: `java -jar scraper.jar -config -` where `otel.jmx.target.system=tomcat` and
   `otel.jmx.service.url=service:jmx:rmi:///jndi/rmi://tomcat:9010/jmxrmi` is written to stdin.
+- environment variables: `OTEL_JMX_TARGET_SYSTEM=tomcat OTEL_JMX_SERVICE_URL=service:jmx:rmi:///jndi/rmi://tomcat:9010/jmxrmi java -jar scraper.jar`
 
-TODO: update this once autoconfiguration is supported
+SDK auto-configuration is being used, so all the configuration options can be set using the java
+properties syntax or the corresponding environment variables.
 
-### Configuration reference
+For example the `otel.jmx.service.url` option can be set with the `OTEL_JMX_SERVICE_URL` environment variable.
 
-TODO
+## Configuration reference
 
-### Extra libraries in classpath
+| config option                     | description                                                                                  |
+|-----------------------------------|----------------------------------------------------------------------------------------------|
+| `otel.jmx.service.url`            | mandatory JMX URL to connect to the remote JVM                                               |
+| `otel.jmx.target.system`          | comma-separated list of systems to monitor, mandatory unless a custom configuration is used  |
+| `otel.jmx.custom.scraping.config` | path to a custom YAML metrics definition, mandatory when `otel.jmx.target.system` is not set |
+| `otel.jmx.username`               | user name for JMX connection                                                                 |
+| `otel.jmx.password`               | password for JMX connection                                                                  |
+
+Supported values for `otel.jmx.target.system`:
+
+| `otel.jmx.target.system` | description           |
+|--------------------------|-----------------------|
+| `activemq`               | Apache ActiveMQ       |
+| `cassandra`              | Apache Cassandra      |
+| `hbase`                  | Apache HBase          |
+| `hadoop`                 | Apache Hadoop         |
+| `jetty`                  | Eclipse Jetty         |
+| `jvm`                    | JVM runtime metrics   |
+| `kafka`                  | Apache Kafka          |
+| `kafka-consumer`         | Apache Kafka consumer |
+| `kafka-producer`         | Apache Kafka producer |
+| `solr`                   | Apache Solr           |
+| `tomcat`                 | Apache Tomcat         |
+| `wildfly`                | Wildfly               |
+
+The following SDK configuration options are also relevant
+
+| config option                 | default value   | description                                                                                                                                                       |
+|-------------------------------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `otel.metric.export.interval` | `1m` (1 minute) | metric export interval, also controls the JMX sampling interval                                                                                                   |
+| `otel.metrics.exporter`       | `otlp`          | comma-separated list of metrics exporters supported values are `otlp` and `logging`, additional values might be provided through extra libraries in the classpath |
+
+In addition to OpenTelemetry configuration, the following Java system properties can be provided
+through the command-line arguments, properties file or stdin and will be propagated to the JVM system properties:
+
+- `javax.net.ssl.keyStore`
+- `javax.net.ssl.keyStorePassword`
+- `javax.net.ssl.trustStore`
+- `javax.net.ssl.trustStorePassword`
+
+Those JVM system properties can't be set through individual environment variables, but they can still
+be set through the standard `JAVA_TOOL_OPTIONS` environment variable using the `-D` prefix.
+
+## Troubleshooting
+
+In order to investigate when and what metrics are being captured and sent, setting the `otel.metrics.exporter`
+configuration option to include `logging` exporter provides log messages when metrics are being exported.
+
+## Extra libraries in classpath
 
 By default, only the RMI JMX connector is provided by the JVM, so it might be required to add extra
 libraries in the classpath when connecting to remote JVMs that are not directly accessible with RMI.
@@ -45,7 +95,7 @@ needs to be used to support `otel.jmx.service.url` = `service:jmx:remote+http://
 When doing so, the `java -jar` command can´t be used, we have to provide the classpath with
 `-cp`/`--class-path`/`-classpath` option and provide the main class file name:
 
-```
+```bash
 java -cp scraper.jar:jboss-client.jar io.opentelemetry.contrib.jmxscraper.JmxScraper <config>
 ```
 

jmx-scraper/src/integrationTest/java/io/opentelemetry/contrib/jmxscraper/JmxScraperContainer.java

@@ -24,7 +24,6 @@ public class JmxScraperContainer extends GenericContainer<JmxScraperContainer> {
   private final String endpoint;
   private final Set<String> targetSystems;
   private String serviceUrl;
-  private int intervalMillis;
   private final Set<String> customYamlFiles;
   private String user;
   private String password;
@@ -44,7 +43,6 @@ public JmxScraperContainer(String otlpEndpoint, String baseImage) {
     this.endpoint = otlpEndpoint;
     this.targetSystems = new HashSet<>();
     this.customYamlFiles = new HashSet<>();
-    this.intervalMillis = 1000;
     this.extraJars = new ArrayList<>();
   }
 
@@ -54,12 +52,6 @@ public JmxScraperContainer withTargetSystem(String targetSystem) {
     return this;
   }
 
-  @CanIgnoreReturnValue
-  public JmxScraperContainer withIntervalMillis(int intervalMillis) {
-    this.intervalMillis = intervalMillis;
-    return this;
-  }
-
   @CanIgnoreReturnValue
   public JmxScraperContainer withRmiServiceUrl(String host, int port) {
     // TODO: adding a way to provide 'host:port' syntax would make this easier for end users
@@ -132,7 +124,8 @@ public void start() {
       throw new IllegalStateException("Missing service URL");
     }
     arguments.add("-Dotel.jmx.service.url=" + serviceUrl);
-    arguments.add("-Dotel.jmx.interval.milliseconds=" + intervalMillis);
+    // always use a very short export interval for testing
+    arguments.add("-Dotel.metric.export.interval=1s");
 
     if (user != null) {
       arguments.add("-Dotel.jmx.username=" + user);

jmx-scraper/src/main/java/io/opentelemetry/contrib/jmxscraper/ArgumentsParsingException.java

@@ -11,4 +11,8 @@ public class ArgumentsParsingException extends Exception {
   public ArgumentsParsingException(String msg) {
     super(msg);
   }
+
+  public ArgumentsParsingException(String msg, Throwable cause) {
+    super(msg, cause);
+  }
 }

jmx-scraper/src/main/java/io/opentelemetry/contrib/jmxscraper/JmxScraper.java

@@ -6,11 +6,14 @@
 package io.opentelemetry.contrib.jmxscraper;
 
 import io.opentelemetry.api.GlobalOpenTelemetry;
-import io.opentelemetry.contrib.jmxscraper.config.ConfigurationException;
 import io.opentelemetry.contrib.jmxscraper.config.JmxScraperConfig;
+import io.opentelemetry.contrib.jmxscraper.config.PropertiesCustomizer;
+import io.opentelemetry.contrib.jmxscraper.config.PropertiesSupplier;
 import io.opentelemetry.instrumentation.jmx.engine.JmxMetricInsight;
 import io.opentelemetry.instrumentation.jmx.engine.MetricConfiguration;
 import io.opentelemetry.instrumentation.jmx.yaml.RuleParser;
+import io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk;
+import io.opentelemetry.sdk.autoconfigure.spi.ConfigurationException;
 import java.io.DataInputStream;
 import java.io.IOException;
 import java.io.InputStream;
@@ -19,9 +22,11 @@
 import java.util.Arrays;
 import java.util.Collections;
 import java.util.List;
+import java.util.Map;
 import java.util.Optional;
 import java.util.Properties;
 import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.logging.Level;
 import java.util.logging.Logger;
 import javax.management.MBeanServerConnection;
 import javax.management.remote.JMXConnector;
@@ -30,8 +35,6 @@ public class JmxScraper {
   private static final Logger logger = Logger.getLogger(JmxScraper.class.getName());
   private static final String CONFIG_ARG = "-config";
 
-  private static final String OTEL_AUTOCONFIGURE = "otel.java.global-autoconfigure.enabled";
-
   private final JmxConnectorBuilder client;
   private final JmxMetricInsight service;
   private final JmxScraperConfig config;
@@ -43,58 +46,77 @@ public class JmxScraper {
    *
    * @param args - must be of the form "-config {jmx_config_path,'-'}"
    */
-  @SuppressWarnings({"SystemOut", "SystemExitOutsideMain"})
+  @SuppressWarnings("SystemExitOutsideMain")
   public static void main(String[] args) {
 
-    // enable SDK auto-configure if not explicitly set by user
-    // TODO: refactor this to use AutoConfiguredOpenTelemetrySdk
-    if (System.getProperty(OTEL_AUTOCONFIGURE) == null) {
-      System.setProperty(OTEL_AUTOCONFIGURE, "true");
-    }
+    // set log format
+    System.setProperty("java.util.logging.SimpleFormatter.format", "%1$tF %1$tT %4$s %5$s%n");
 
     try {
-      JmxScraperConfig config =
-          JmxScraperConfig.fromProperties(parseArgs(Arrays.asList(args)), System.getProperties());
-      // propagate effective user-provided configuration to JVM system properties
-      // this also enables SDK auto-configuration to use those properties
-      config.propagateSystemProperties();
+      Properties argsConfig = parseArgs(Arrays.asList(args));
+      propagateToSystemProperties(argsConfig);
+
+      // auto-configure and register SDK
+      PropertiesCustomizer configCustomizer = new PropertiesCustomizer();
+      AutoConfiguredOpenTelemetrySdk.builder()
+          .addPropertiesSupplier(new PropertiesSupplier(argsConfig))
+          .addPropertiesCustomizer(configCustomizer)
+          .setResultAsGlobal()
+          .build();
+
+      JmxScraperConfig scraperConfig = configCustomizer.getScraperConfig();
+
+      long exportSeconds = scraperConfig.getSamplingInterval().toMillis() / 1000;
+      logger.log(Level.INFO, "metrics export interval (seconds) =  " + exportSeconds);
 
       JmxMetricInsight service =
           JmxMetricInsight.createService(
-              GlobalOpenTelemetry.get(), config.getIntervalMilliseconds());
-      JmxConnectorBuilder connectorBuilder = JmxConnectorBuilder.createNew(config.getServiceUrl());
+              GlobalOpenTelemetry.get(), scraperConfig.getSamplingInterval().toMillis());
+      JmxConnectorBuilder connectorBuilder =
+          JmxConnectorBuilder.createNew(scraperConfig.getServiceUrl());
 
-      Optional.ofNullable(config.getUsername()).ifPresent(connectorBuilder::withUser);
-      Optional.ofNullable(config.getPassword()).ifPresent(connectorBuilder::withPassword);
+      Optional.ofNullable(scraperConfig.getUsername()).ifPresent(connectorBuilder::withUser);
+      Optional.ofNullable(scraperConfig.getPassword()).ifPresent(connectorBuilder::withPassword);
 
-      JmxScraper jmxScraper = new JmxScraper(connectorBuilder, service, config);
+      JmxScraper jmxScraper = new JmxScraper(connectorBuilder, service, scraperConfig);
       jmxScraper.start();
-
+    } catch (ConfigurationException e) {
+      logger.log(Level.SEVERE, "ERROR: invalid configuration ", e);
+      System.exit(1);
     } catch (ArgumentsParsingException e) {
-      System.err.println("ERROR: " + e.getMessage());
-      System.err.println(
+      logger.log(Level.SEVERE, "ERROR: invalid configuration provided through arguments", e);
+      logger.info(
           "Usage: java -jar <path_to_jmxscraper.jar> "
               + "-config <path_to_config.properties or - for stdin>");
       System.exit(1);
-    } catch (ConfigurationException e) {
-      System.err.println(e.getMessage());
-      System.exit(1);
     } catch (IOException e) {
-      System.err.println("Unable to connect " + e.getMessage());
+      logger.log(Level.SEVERE, "Unable to connect ", e);
       System.exit(2);
     } catch (RuntimeException e) {
-      e.printStackTrace(System.err);
+      logger.log(Level.SEVERE, e.getMessage(), e);
       System.exit(3);
     }
   }
 
+  // package private for testing
+  static void propagateToSystemProperties(Properties properties) {
+    for (Map.Entry<Object, Object> entry : properties.entrySet()) {
+      String key = entry.getKey().toString();
+      String value = entry.getValue().toString();
+      if (key.startsWith("javax.net.ssl.keyStore") || key.startsWith("javax.net.ssl.trustStore")) {
+        if (System.getProperty(key) == null) {
+          System.setProperty(key, value);
+        }
+      }
+    }
+  }
+
   /**
    * Create {@link Properties} from command line options
    *
    * @param args application commandline arguments
    */
-  static Properties parseArgs(List<String> args)
-      throws ArgumentsParsingException, ConfigurationException {
+  static Properties parseArgs(List<String> args) throws ArgumentsParsingException {
 
     if (args.isEmpty()) {
       // empty properties from stdin or external file
@@ -116,23 +138,24 @@ static Properties parseArgs(List<String> args)
     }
   }
 
-  private static Properties loadPropertiesFromStdin() throws ConfigurationException {
+  private static Properties loadPropertiesFromStdin() throws ArgumentsParsingException {
     Properties properties = new Properties();
     try (InputStream is = new DataInputStream(System.in)) {
       properties.load(is);
       return properties;
     } catch (IOException e) {
-      throw new ConfigurationException("Failed to read config properties from stdin", e);
+      throw new ArgumentsParsingException("Failed to read config properties from stdin", e);
     }
   }
 
-  private static Properties loadPropertiesFromPath(String path) throws ConfigurationException {
+  private static Properties loadPropertiesFromPath(String path) throws ArgumentsParsingException {
     Properties properties = new Properties();
     try (InputStream is = Files.newInputStream(Paths.get(path))) {
       properties.load(is);
       return properties;
     } catch (IOException e) {
-      throw new ConfigurationException("Failed to read config properties file: '" + path + "'", e);
+      throw new ArgumentsParsingException(
+          "Failed to read config properties file: '" + path + "'", e);
     }
   }