deprecation and removal process of JMX Gatherer

[SylvainJuge]

The goal of this issue is to define and discuss the steps of JMX Gatherer deprecation and removal.

current status (as of writing)

• JMX Gatherer is documented as deprecated
• JMX Scraper provides a replacement of JMX Gatherer.
• JMX Scraper provides a legacy metrics definitions set that is very close to JMX Gatherer (see otel.jmx.target.source in config)
• JMX Scraper provides a migration guide
• otel collector jmxreceiver can use JMX Gatherer (default) or JMX Scraper (opt-in)

desired end goal

• users of JMX Gatherer are aware the component is being deprecated and should be able to migrate to JMX Scraper
• users of JMX Gatherer through the collector jmxreceiver could have a multi-step migration process to minimize migration burden
• users should always be able to opt-out by sticking to an older version of JMX Gatherer and/or using otel.jmx.target.source=legacy with JMX Scraper.
• JMX Gatherer component is removed from code and is no longer published on release.

things to discuss

• for jmxreceiver should we do the migration in one step (big-bang) or two steps (implementation+legacy first, then aligned metrics) ?
• for direct usages of JMX Gatherer, should we use an obtrusive way to ensure users are aware the component is being removed ?

steps

• step 1 : in jmxreceiver switch default implementation to JMX Scraper with otel.jmx.target.source=legacy
  • users can opt-out by switching to JMX Gatherer implementation (jmxreceiver should provide a documented way to do that)
  • users can opt-in to aligned metrics by setting otel.jmx.target.source to instrumentation or auto to expose themselves to more recent (but moving) metrics definitions until JMX implementation : feature parity for target systems opentelemetry-java-instrumentation#12158 is complete
• wait ~ 3 months for feedback on step 1.
• wait for completion of JMX implementation : feature parity for target systems opentelemetry-java-instrumentation#12158, which implies otel.jmx.target.source set to auto or instrumentation would be equivalent.
• step 2 : in jmxreceiver set otel.jmx.target.source=auto
  • users can opt-out by switching to JMX Gatherer implementation
  • users can partially opt-out by switching to otel.jmx.target.source=legacy
  • communicate breaking change as all captured metrics will change
• wait ~ 3 months for feedback on step 2.
• step 3 : implement an obtrusive way to notify users of JMX Gatherer
  • we can add a gentle warning with a link to migration docs, but unlikely to get noticed
  • we can add an extra mandatory CLI argument making it fail otherwise
  • users of custom groovy metrics definitions need to be aware they will have to migrate
  • users can always opt-out by adding the extra argument or sticking to an older version of JMX Gatherer
• wait ~ 3 months for feedback on step 3.
• step 4 : remove JMX Gatherer from code and published artifacts
  • users can always opt-out by sticking to the latest published (or older) version of JMX Gatherer
  • we can close related issues and remove component:jmx-metrics label.

[SylvainJuge]

cc @robsunday for JMX Scraper
cc @atoulme @rogercoll as owners of jmxreceiver collector component

[rogercoll]

for jmxreceiver should we do the migration in one step (big-bang) or two steps (implementation+legacy first, then aligned metrics) ?
for direct usages of JMX Gatherer, should we use an obtrusive way to ensure users are aware the component is being removed ?

For the jmxreceiver I would propose using the feature gates strategy. On alpha stage, the feature gate should default to the jmx-scraper and a warning message should be logged if jmx-gatherer is enabled. On beta, make the jmx-scraper the default JAR file, and on stable remove the jmx-gatherer support.

[SylvainJuge]

For the jmxreceiver I would propose using the feature gates strategy. On alpha stage, the feature gate should default to the jmx-scraper and a warning message should be logged if jmx-gatherer is enabled. On beta, make the jmx-scraper the default JAR file, and on stable remove the jmx-gatherer support.

Using feature gates seems a good fit here, maybe having two of them: one for the switch by default to the jmx scraper implementation (step 1), then another one to switch the otel.jmx.target.source to auto by defaul (step 2).

This plan does not yet include removing support for gatherer in collector, but we could handle this when removing artifact publication and build (step 4), which could either be handled as an independent feature gate or by extending the meaning of the one added for switching default implementation (finding the right feature gate name might be key for this).