Commit e144053
Add missing JMX metrics for ContainerInsights. (#898)
### Description of changes
* Added missing metrics from the ContainerInsights
[dashboard](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-Prometheus-metrics.html#ContainerInsights-Prometheus-metrics-jmx)
not included in the
[agent](https://github.com/aws-observability/aws-otel-java-instrumentation/blob/c39b76830931899a678f72d502dafe33329adde5/instrumentation/jmx-metrics/src/main/resources/jmx/rules/jvm.yaml)'s
solution for JMX.
* Compared the current spec files for JVM and Tomcat with the dashboard
to see what was missing.
* Set up an EC2 cluster, a Tomcat application, and JMXTerm to find the
bean and metric names for the missing metrics.
* Added these to the spec file based on appropriate types and the naming
convention from
[OTel](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/javaagent/README.md).
* Then, the ADOT Jar image was built and tested with the
`amazon-cloudwatch-agent-operator` and `helm-charts` on an EKS cluster
with a custom Tomcat deployment, which correctly produced the correct
metrics:
* `./scripts/local_patch.sh && ./gradlew build`
* `docker build --platform linux/amd64 -t adot-autoinstrumentation-java
.`
* `[add docker image to ECR]`
### Manual testing
For testing, I decided to use this built image with the CloudWatch Agent
to see if these metrics are able to be emitted and tracked on the CW
console.
#### Configuration
```
{
"metrics": {
"namespace": "tomcat",
"metrics_collected": {
"jmx": {
"jvm": {
"measurement": [
"jvm.system.swap.space.total",
"jvm.system.swap.space.free",
"jvm.system.physical.memory.total",
"jvm.system.physical.memory.free",
"jvm.system.available.processors",
"jvm.system.cpu.utilization",
"jvm.open_file_descriptor.count",
"jvm.daemon_threads.count",
"jvm.threads.count"
]
},
"tomcat": {
"measurement": [
"tomcat.rejected_sessions",
"tomcat.sessions"
]
}
}
}
}
}
```
#### CW Console
<img width="949" alt="Screenshot 2024-10-04 at 11 08 51 AM"
src="https://github.com/user-attachments/assets/e9f3869b-f17d-4d17-81b5-0c0b97272012">
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Mengyi Zhou (bjrara) <[email protected]>1 parent 51098f9 commit e144053
File tree
3 files changed
+48
-10
lines changed- instrumentation/jmx-metrics/src/main/resources
- jmx/rules
3 files changed
+48
-10
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
Lines changed: 34 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | | - | |
79 | 78 | | |
80 | 79 | | |
81 | 80 | | |
82 | | - | |
| 81 | + | |
83 | 82 | | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
84 | 86 | | |
85 | | - | |
86 | 87 | | |
87 | 88 | | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
88 | 113 | | |
89 | | - | |
| 114 | + | |
90 | 115 | | |
91 | 116 | | |
92 | 117 | | |
93 | | - | |
| 118 | + | |
94 | 119 | | |
95 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
Lines changed: 10 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
| 7 | + | |
6 | 8 | | |
7 | 9 | | |
8 | 10 | | |
9 | | - | |
10 | | - | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
12 | 15 | | |
13 | 16 | | |
14 | 17 | | |
| |||
68 | 71 | | |
69 | 72 | | |
70 | 73 | | |
| 74 | + | |
| 75 | + | |
71 | 76 | | |
72 | 77 | | |
73 | 78 | | |
74 | | - | |
75 | | - | |
76 | 79 | | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
77 | 83 | | |
78 | 84 | | |
79 | 85 | | |
| |||
0 commit comments