Thread and Thread pool metric automatic instrumentation

[Falmarri]

Is your feature request related to a problem? Please describe.

Getting thread metrics is pretty important, and is supported by most APMs. Before I can get all of my service owners to switch off of whatever APM they're currently using, I need at least a reasonable degree of feature parity in the metrics.

Describe the solution you'd like

Thread metrics available to export to otel and/or serve via prometheus. Including

• number of current threads total
• number of threads used out of a thread pool
• number of runnables executed by that pool
• pool queue size

Describe alternatives you've considered

No response

Additional context

No response

[pepeshore]

I have implemented the collection of thread pool metrics in my forked repository.
including the following metrics:

• thread_pool_core_pool_size
• thread_pool_max_pool_size
• thread_pool_active_thread_count
• thread_pool_current_thread_count
• thread_pool_max_thread_count
• thread_pool_scheduled_task_count
• thread_pool_completed_task_count
• thread_pool_rejected_task_count
• thread_pool_queue_size
  labels:
• thread.name.pattern(e.g. http-nio-8080-*)
• thread.pool.class.name(e.g. java.util.concurrent.ThreadPoolExecutor)

if this is helpful, i'd like to create a pr

[AlchemyDing]

Refer to micrometer

[AlchemyDing]

I think this is necessary, but the name of metrics needs to be considered.
cc @trask @laurit @steverao

[steverao]

It looks like reasonable feature, @open-telemetry/java-instrumentation-approvers WDYT?

[laurit]

I think it is a reasonable request, but it is too vague. It would be helpful to know exactly which thread pools the users are interested it. It should already be possible to monitor many thread pools using https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/jmx-metrics/javaagent though users may need to write the configuration themself.

[pepeshore]

I think it is a reasonable request, but it is too vague. It would be helpful to know exactly which thread pools the users are interested it. It should already be possible to monitor many thread pools using https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/jmx-metrics/javaagent though users may need to write the configuration themself.

Most user-defined thread pools cannot be monitored through JMX metrics. And why do you think it si too vague, you don't know which thread pool should be monitored?

[laurit]

And why do you think it si too vague, you don't know which thread pool should be monitored?

No, I don't. There are many ways one could implement a thread pool, there won't be a catch all solution. Even something seemingly simple, like instrumenting ThreadPoolExecutor, immediately gets complicated when you consider that you somehow need to identify which ThreadPoolExecutor the metrics belong to.

[pepeshore]

No, I don't. There are many ways one could implement a thread pool

Indeed, there are many implementations of thread pools. However, it is sufficient to monitor the mainstream thread pools. Similar to the existing monitoring of connection pools, there are also various implementations of connection pools, but we only monitor a part of them, such as c3p0, dbcp, and so on.

[pepeshore]

Even something seemingly simple, like instrumenting ThreadPoolExecutor, immediately gets complicated when you consider that you somehow need to identify which ThreadPoolExecutor the metrics belong to.

I considered this issue as well. When I added two dimensions to the metrics related to thread pools, namely thread.name.pattern (e.g., http-nio-8080-*) and thread.pool.create.stack (indicating the method stack that created this thread pool), the association between a metric and its thread pool became clear.

[pepeshore]

@laurit Hi laurit，What do you think about my opinion？I do think is't a useful feature

[laurit]

@pepeshore sure, a PR would be welcome

[pepeshore]

@pepeshore sure, a PR would be welcome

ok，assign this issue to me pls