Skip to content

Conversation

@mitali-salvi
Copy link
Contributor

@mitali-salvi mitali-salvi commented Oct 3, 2024

Description of changes:

  1. The operator will inject init-containers, open up ports on agent service/container if the following annotation is attached to a K8s workload - cloudwatch.aws.amazon.com/inject-jmx-{targetSystem} where targetSystem includes jvm, "tomcat", "kafka", "kafka-consumer", "kafka-producer"
  2. Operator will inject JMX specific environment variables in the pod -
    "OTEL_EXPORTER_OTLP_PROTOCOL"= "http/protobuf"
    "OTEL_METRICS_EXPORTER"="none"
    "OTEL_LOGS_EXPORTER"="none"
    "OTEL_AWS_JMX_EXPORTER_METRICS_ENDPOINT="http://cloudwatch-agent.amazon-cloudwatch:4314/v1/metrics"
    "OTEL_JMX_TARGET_SYSTEM"="jvm,tomcat"
  3. Operator will open port 4314 on cloudwatch-agent container and service for reading metrics over HTTP protocol when the agent config has JMX enabled in the metrics section OR JMX-Container Insights is enabled in logs section
  4. This PR is will use the custom JMX exporter to export JMX metrics - Implementing OtelJMX metrics exporter for JMX metrics insights aws-observability/aws-otel-java-instrumentation#901

Testing

  1. Unit testing
  2. Manual testing using the amazon-cloudwatch-observability helm-chart
    i. Annotate a pod deploying a sprint boot project with instrumentation.opentelemetry.io/inject-java: "true" cloudwatch.aws.amazon.com/inject-jmx-jvm: "true" and deploy on the cluster
    ii. Verified environment variables and init container are added to the pod
    iii. Verified cloudwatch-agent service and container has the port 4314 open when the agent config has JMX enabled
    iv. Verified jmx metrics are being published to CloudWatch

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Paramadon
Paramadon previously approved these changes Oct 9, 2024
musa-asad
musa-asad previously approved these changes Oct 11, 2024
@mitali-salvi
Copy link
Contributor Author

CloudWatch agent container port

│ Containers:                                                                                                                                                                                                                                               
│   otc-container:                                                                                                                                                                                                                                          
│     Container ID:   containerd://09823f59743f80abdf890a11dca9a7cd6f164cda4be1f2fb9e09fc5bb55737db                                                                                                                                                                                                                                                                  
│     Port:           4314/TCP                                                                                                                                                                                                                              
│     Host Port:      4314/TCP 

Cloudwatch agent service

 Port:              jmx-http  4314/TCP

@mitali-salvi mitali-salvi merged commit c277a41 into main Oct 14, 2024
8 checks passed
@mitali-salvi mitali-salvi deleted the jmx-merge branch October 14, 2024 16:21
okankoAMZ pushed a commit to okankoAMZ/amazon-cloudwatch-agent-operator that referenced this pull request Oct 31, 2024
okankoAMZ added a commit that referenced this pull request Nov 5, 2024
* Adding support for NodeJS auto instrumentation and integ tests (#220)

* Support configurable resources for NodeJS. (#225)

* Supporting JMX annotations (#240)

* Add support for a supplemental YAML configuration for the CloudWatchAgent (#241)

* Changed naming for OTLP container ports from agent JSON (#252)

* Updated Release Notes for 1.8.0 (#251)

* Adjust EKS add-on integration test service count expectations (#256)

* Add integration tests for JMX. (#250)

* Implemented Target Allocator Container (#214)

* Implemented TargetAllocator resource deployments. (#208)

* Update cmd/amazon-cloudwatch-agent-target-allocator/config/config.go

Co-authored-by: Musa <[email protected]>

* Update internal/config/main.go

Co-authored-by: Musa <[email protected]>

---------

Co-authored-by: Parampreet Singh <[email protected]>
Co-authored-by: Musa <[email protected]>
Co-authored-by: Mitali Salvi <[email protected]>
Co-authored-by: Jeffrey Chien <[email protected]>
okankoAMZ added a commit that referenced this pull request Nov 26, 2024
* Implemented Target Allocator Container (#214)

* Merge `main` into `target-allocator` (#232)

NodeJS merging-in from main

* Supporting K8s 1.31 (#222) (#236)

Co-authored-by: Mitali Salvi <[email protected]>

* Implemented TargetAllocator resource deployments. (#208)

* Adding target-allocator label to service selectors (#242)

* Target allocator TLS Implementation  (#239)

* Ta https server (#2921)

* Added https server, tests, secret marshalling


---------

Co-authored-by: ItielOlenick <[email protected]>

* [Target Allocator] Enable Deployment and Daemonset modes for Agent (#253)

* Changes error to warning

* [CI/CD] Add Target Allocator(TA) Build to Build and Upload Workflow (#247)

* edited workflow

* Clean up  managed resources when disabled (#255)

* Reconciler now removes un-used managed resources for CWA collector

* remove pprof endpoint (#260)

* [TA] One service per Target Allocator  (#259)

* added one-service per TA

* Setup cert-watcher for TA server cert (#264)

* [TA] Target Allocator TLS Unit-tests (#265)

* TLS tests

* Injecting Prometheus path if not specified in agent config (#258)

* Injecting Prom path if it doesn't exist

* Rebasing Target Allocator Branch to Main  (#266)

* Adding support for NodeJS auto instrumentation and integ tests (#220)

* Support configurable resources for NodeJS. (#225)

* Supporting JMX annotations (#240)

* Add support for a supplemental YAML configuration for the CloudWatchAgent (#241)

* Changed naming for OTLP container ports from agent JSON (#252)

* Updated Release Notes for 1.8.0 (#251)

* Adjust EKS add-on integration test service count expectations (#256)

* Add integration tests for JMX. (#250)

* Implemented Target Allocator Container (#214)

* Implemented TargetAllocator resource deployments. (#208)

* Update cmd/amazon-cloudwatch-agent-target-allocator/config/config.go

Co-authored-by: Musa <[email protected]>

* Update internal/config/main.go

Co-authored-by: Musa <[email protected]>

---------

Co-authored-by: Parampreet Singh <[email protected]>
Co-authored-by: Musa <[email protected]>
Co-authored-by: Mitali Salvi <[email protected]>
Co-authored-by: Jeffrey Chien <[email protected]>


---------

Co-authored-by: Musa <[email protected]>
Co-authored-by: Mitali Salvi <[email protected]>
Co-authored-by: ItielOlenick <[email protected]>
Co-authored-by: Kaushik Surya <[email protected]>
Co-authored-by: Parampreet Singh <[email protected]>
Co-authored-by: Jeffrey Chien <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants