deprecate jmx gatherer + document migration (#2034)

Author: SylvainJuge

jmx-metrics/README.md

@@ -1,5 +1,7 @@
 # JMX Metric Gatherer
 
+**Deprecation notice**: the JMX Metric Gatherer is deprecated and replaced by [JMX Scraper](../jmx-scraper/), see [migration instructions](../jmx-scraper/README.md#migration-from-jmx-gatherer).
+
 This utility provides an easy framework for gathering and reporting metrics based on queried
 MBeans from a JMX server.  It loads included and/or custom Groovy scripts and establishes a helpful,
 bound `otel` object with methods for obtaining MBeans and constructing OpenTelemetry instruments:

jmx-scraper/README.md

@@ -4,8 +4,7 @@ This utility provides a way to query JMX metrics and export them to an OTLP endp
 The JMX MBeans and their metric mappings are defined in YAML and reuse implementation from
 [jmx-metrics instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/jmx-metrics).
 
-This is currently a work-in-progress component not ready to be used in production.
-The end goal is to provide an alternative to the [JMX Gatherer](../jmx-metrics/README.md) utility.
+This is an alternative to the [JMX Gatherer](../jmx-metrics/README.md) utility.
 
 ## Usage
 
@@ -136,6 +135,45 @@ When doing so, the `java -jar` command can´t be used, we have to provide the cl
 java -cp scraper.jar:jboss-client.jar io.opentelemetry.contrib.jmxscraper.JmxScraper <config>
 ```
 
+## Migration from JMX Gatherer
+
+The JMX Scraper aims to replace the [JMX Gatherer](../jmx-metrics) tool and thus share most features
+and configuration with it.
+
+Features not supported:
+
+- Define and capture metrics with custom Groovy definitions with `otel.jmx.groovy.script`, this is now replaced with YAML and `otel.jmx.config` configuration option.
+- Ability to export to prometheus collector, only the OTLP exporter is included.
+
+The YAML-based implementation is provided by [java instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/jmx-metrics)
+and thus should be used for syntax details and documentation.
+
+Like with the JMX Gatherer, the selection of provided metrics to use is still done with `otel.jmx.target.system` configuration option.
+
+However, there is now two distinct sets of metrics to select from using the `otel.jmx.target.source` configuration option:
+
+- `legacy`: [metrics definitions](./src/main/resources) equivalent to JMX Gatherer definitions to help transition and preserve compatibility
+- `instrumentation`: [metrics definitions inherited from instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/library/src/main/resources/jmx/rules/), which is now used as a reference for JMX metrics, those also aim to provide better alignment with [metrics semantic conventions](https://opentelemetry.io/docs/specs/semconv/general/metrics/).
+
+In both cases, the metrics definitions themselves are embedded in the JMX Scraper binary, thus they
+will only change if the release version of the JMX Scraper binary changes.
+
+By default, `otel.jmx.target.source` is `auto`, which means for each value of `otel.jmx.target.system`:
+
+- Metrics definitions from instrumentation will be used by default, if available.
+- Legacy metrics definitions equivalent to JMX Gatherer will be used as fallback.
+- Whenever new metrics definitions are being added or modified in instrumentation, those newer definitions will be used.
+
+There are multiple possible strategies depending on the ability or willingness to embrace change in metrics definitions:
+
+- To preserve maximum compatibility, using `legacy` is the recommended option, however it means to not benefit from future updates and contributions.
+- To only get the most recent definitions, using `instrumentation` ensures that none of the legacy definitions is used, only the reference from instrumentation closer to semconv recommendations, those could still evolve over time.
+- To embrace reference definitions whenever they become available, using `auto` is the recommended option, however it means the metrics produced could change when updating the version of JMX Scraper.
+- To handle more complex migration strategies or for tight control of metrics definitions, using copies of the YAML metrics definitions and providing them explicitly with `otel.jmx.config` is the recommended option.
+
+When using `otel.target.source` = `auto` or `legacy`, one or more legacy definitions might be used. If strict compatibility with metrics produced by JMX Gatherer is required it is recommended to review
+the [legacy metrics definitions YAML files](./src/main/resources/) as they contain comments on the minor differences with JMX Gatherer Groovy definitions.
+
 ## Component owners
 
 - [Jason Plumb](https://github.com/breedx-splk), Splunk