Commit b60ba73
authored
feat: [Java] EMF Exporter Implementation (#1209)
*Description of changes:*
Java Version of these PRs:
-
aws-observability/aws-otel-python-instrumentation#382
-
aws-observability/aws-otel-python-instrumentation#409
-
aws-observability/aws-otel-python-instrumentation#410
-
aws-observability/aws-otel-python-instrumentation#434
This PR introduces the complete CloudWatch EMF (Embedded Metric Format)
exporter implementation for sending OpenTelemetry metrics directly to
CloudWatch without requiring a Collector or Agent.
In order to enable this exporter, users MUST set the following
environment variables:
- `OTEL_METRICS_EXPORTER=awsemf` -
-
`OTEL_EXPORTER_OTLP_LOGS_HEADERS=x-aws-log-group=<log-group-name>,x-aws-log-stream=<log-stream-name>,
x-aws-metric-namespace=<namespace>`
- `AWS_REGION=<region>` OR `AWS_DEFAULT_REGION=<region>`
This PR includes:
- EMF MetricRecord translation for for unified representation of all
OTel metric types with log creation with unit mapping from OpenTelemetry
to CloudWatch-compatible units
- Automatic log group and stream creation with retry logic
- Supported CloudWatch Logs integration with batching and constraint
handling (256KB event limit, 1MB request limit, timestamp limits)
- Support for metric grouping by attributes and timestamps for EMF log
generation
- Support for Gauge, Sum, Histogram, and ExponentialHistogram metric
types
**TODO**:
- On the next PR, will integrate the Console EMF exporter into AWS
Lambda environments to validate EMF log formatting and ensure consistent
behavior in Lambda runtime:
aws-observability/aws-otel-python-instrumentation#437
**Testing**:
- Added unit tests to validate EMF exporter configuration scenarios,
including parameterized tests for both valid configurations (supporting
AWS_REGION and AWS_DEFAULT_REGION) and invalid configurations (missing
headers, wrong exporter type, missing region). The tests ensure the EMF
exporter is correctly enabled only when all required environment
variables are properly configured.
- Manual end to end testing with the following environment variables to
ensure the EMF logs show up:
- `AWS_REGION=us-east-1`
- `OTEL_METRICS_EXPORTER=awsemf`
-
`OTEL_EXPORTER_OTLP_LOGS_HEADERS=x-aws-log-group=test,x-aws-log-stream=default,x-aws-metric-namespace=testNamespace`
-
`OTEL_RESOURCE_ATTRIBUTES=service.name=testService,aws.log.group.names=test,cloud.resource_id=agent-12345`
- `OTEL_LOGS_EXPORTER=none`
- `OTEL_TRACES_EXPORTER=none`
Example EMF log emitted:
```
{
"otel.resource.process.command_args": "[/Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home/bin/java, -javaagent:/Users/liustve/aws-otel-java-instrumentation/otelagent/build/libs/aws-opentelemetry-agent-2.18.0-SNAPSHOT.jar, -Dfile.encoding=UTF-8, -Dsun.stdout.encoding=UTF-8, -Dsun.stderr.encoding=UTF-8, -jar, /Users/liustve/aws-otel-java-instrumentation/sample-apps/springboot/build/libs/springboot-2.11.0-SNAPSHOT.jar]",
"otel.resource.host.arch": "aarch64",
"otel.resource.host.name": "7cf34dd812df",
"otel.resource.service.instance.id": "a0399d3c-b856-43ae-b374-fe66dee41ce8",
"otel.resource.aws.log.group.names": "test",
"jvm.class.unloaded": 1,
"otel.resource.service.name": "testSErvice",
"_aws": {
"CloudWatchMetrics": [
{
"Metrics": [
{
"Unit": "Count",
"Name": "jvm.cpu.count"
},
{
"Unit": "Count",
"Name": "jvm.class.loaded"
},
{
"Name": "jvm.cpu.recent_utilization"
},
{
"Unit": "Seconds",
"Name": "jvm.cpu.time"
},
{
"Unit": "Count",
"Name": "jvm.class.count"
},
{
"Unit": "Count",
"Name": "jvm.class.unloaded"
}
],
"Namespace": "testNamespace"
}
],
"Timestamp": 1758761023497
},
"otel.resource.cloud.resource_id": "agent-12345",
"jvm.class.count": 14264,
"Version": "1",
"otel.resource.process.pid": "46822",
"otel.resource.os.description": "Mac OS X 15.6.1",
"otel.resource.telemetry.distro.name": "opentelemetry-java-instrumentation",
"otel.resource.os.type": "darwin",
"otel.resource.telemetry.sdk.name": "opentelemetry",
"otel.resource.telemetry.distro.version": "2.18.0-aws-SNAPSHOT",
"otel.resource.process.runtime.description": "Amazon.com Inc. OpenJDK 64-Bit Server VM 21.0.8+9-LTS",
"otel.resource.process.runtime.version": "21.0.8+9-LTS",
"jvm.cpu.recent_utilization": 0,
"otel.resource.process.executable.path": "/Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home/bin/java",
"otel.resource.telemetry.sdk.version": "1.52.0",
"jvm.cpu.count": 12,
"jvm.class.loaded": 14265,
"otel.resource.process.runtime.name": "OpenJDK Runtime Environment",
"otel.resource.telemetry.sdk.language": "java",
"jvm.cpu.time": 7.756965
}
```
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.1 parent 2bde6be commit b60ba73
File tree
17 files changed
+2918
-483
lines changed- awsagentprovider
- src
- main/java/software/amazon/opentelemetry/javaagent/providers
- exporter/aws/metrics
- common
- emitter
- test/java/software/amazon/opentelemetry/javaagent/providers
- exporter
- aws/metrics
- otlp/aws
17 files changed
+2918
-483
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| 56 | + | |
| 57 | + | |
56 | 58 | | |
57 | 59 | | |
58 | 60 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
25 | 26 | | |
26 | 27 | | |
27 | | - | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
31 | 63 | | |
32 | 64 | | |
33 | 65 | | |
| |||
61 | 93 | | |
62 | 94 | | |
63 | 95 | | |
64 | | - | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
65 | 99 | | |
66 | 100 | | |
67 | 101 | | |
| 102 | + | |
| 103 | + | |
68 | 104 | | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
| 105 | + | |
| 106 | + | |
83 | 107 | | |
84 | | - | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
85 | 111 | | |
86 | 112 | | |
87 | 113 | | |
| |||
168 | 194 | | |
169 | 195 | | |
170 | 196 | | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
171 | 219 | | |
Lines changed: 67 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| 68 | + | |
| 69 | + | |
68 | 70 | | |
69 | 71 | | |
70 | 72 | | |
| |||
86 | 88 | | |
87 | 89 | | |
88 | 90 | | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
89 | 94 | | |
90 | 95 | | |
91 | 96 | | |
| |||
103 | 108 | | |
104 | 109 | | |
105 | 110 | | |
| 111 | + | |
106 | 112 | | |
107 | 113 | | |
108 | 114 | | |
| |||
132 | 138 | | |
133 | 139 | | |
134 | 140 | | |
135 | | - | |
| 141 | + | |
136 | 142 | | |
137 | 143 | | |
138 | 144 | | |
| |||
161 | 167 | | |
162 | 168 | | |
163 | 169 | | |
| 170 | + | |
164 | 171 | | |
165 | 172 | | |
166 | 173 | | |
| |||
171 | 178 | | |
172 | 179 | | |
173 | 180 | | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
174 | 190 | | |
175 | 191 | | |
176 | 192 | | |
| |||
190 | 206 | | |
191 | 207 | | |
192 | 208 | | |
193 | | - | |
| 209 | + | |
194 | 210 | | |
195 | 211 | | |
196 | 212 | | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
197 | 221 | | |
198 | 222 | | |
199 | 223 | | |
| |||
394 | 418 | | |
395 | 419 | | |
396 | 420 | | |
397 | | - | |
398 | 421 | | |
399 | 422 | | |
400 | 423 | | |
| |||
434 | 457 | | |
435 | 458 | | |
436 | 459 | | |
437 | | - | |
| 460 | + | |
438 | 461 | | |
439 | 462 | | |
440 | 463 | | |
| |||
480 | 503 | | |
481 | 504 | | |
482 | 505 | | |
483 | | - | |
| 506 | + | |
484 | 507 | | |
485 | 508 | | |
486 | 509 | | |
| |||
509 | 532 | | |
510 | 533 | | |
511 | 534 | | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
512 | 574 | | |
513 | 575 | | |
514 | 576 | | |
| |||
Lines changed: 73 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
0 commit comments