Add docs for adothealth monitoring (#224)

* added docs for ADOT health monitoring

* Update index.md - Added screen-shots of the dashboard

* Update index.md -Corrected some typos and formatting

* Update index.md - Made corrections based on comments

* Update index.md

---------

Co-authored-by: Mathews <[email protected]>

Author: RAMathews

docs/adothealth/index.md

@@ -0,0 +1,54 @@
+# Monitoring ADOT collector health
+
+The OpenTelemetry collector produces metrics to monitor the entire pipeline. In the [EKS monitoring module](https://aws-observability.github.io/terraform-aws-observability-accelerator/eks/), we have enabled those metrics by default with the AWS Distro for OpenTelemetry (ADOT) collector. You get a Grafana dashboard named `OpenTelemetry Health Collector`. This dashboard shows useful telemetry information about the ADOT collector itself which can be helpful when you want to troubleshoot any issues with the collector or understand how much resources the collector is consuming.
+
+!!!note
+    The dashboard and metrics used are not specific to Amazon EKS, but applicable to any environment running an OpenTelemetry collector.
+
+Below diagram shows an example data flow and the components in an ADOT collector:
+
+![ADOTCollectorComponents](https://github.com/RAMathews/terraform-aws-observability-accelerator/assets/114662591/1db25d84-c1ca-4468-bb0d-42c8bafd1942)
+
+In this dashboard, there are five sections. Each section has [metrics](https://aws-observability.github.io/observability-best-practices/guides/operational/adot-at-scale/operating-adot-collector/#collecting-health-metrics-from-the-collector) relevant to the various [components](https://opentelemetry.io/docs/demo/collector-data-flow-dashboard/#data-flow-overview) of the AWS Distro for OpenTelemetry (ADOT) collector :
+
+### Receivers
+Shows the receiver’s accepted and refused rate/count of spans and metric points that are pushed into the telemetry pipeline.
+
+### Processors
+Shows the accepted and refused rate/count of spans and metric points pushed into next component in the pipeline. The batch metrics can help to understand how often metrics are sent to exporter and the batch size.
+
+![receivers_processors](https://github.com/RAMathews/terraform-aws-observability-accelerator/assets/114662591/9a2edc27-9472-4a58-a244-d69f2bc7f41f)
+
+### Exporters
+Shows the exporter’s accepted and refused rate/count of spans and metric points that are pushed to any of the destinations. It also shows the size and capacity of the retry queue. These metrics can be used to understand if the collector is having issues in sending trace or metric data to the destination configured.
+
+![exporters](https://github.com/RAMathews/terraform-aws-observability-accelerator/assets/114662591/77e20ac5-64bb-42ca-9db6-4d13ca7b27de)
+
+### Collectors
+Shows the collector’s operational metrics (Memory, CPU, uptime). This can be used to understand how much resources the collector is consuming.
+
+![collectors](https://github.com/RAMathews/terraform-aws-observability-accelerator/assets/114662591/25151edd-6132-479a-9331-71aa69a91d5e)
+
+### Data Flow
+Shows the metrics and spans data flow through the collector’s components.
+
+![dataflow](https://github.com/RAMathews/terraform-aws-observability-accelerator/assets/114662591/61fe684d-8ed3-4645-9210-f16158442b7d)
+
+!!!note
+    To read more about the metrics used, and the dashboard use, visit the upstream documentation [here](https://opentelemetry.io/docs/demo/collector-data-flow-dashboard/).
+
+## Deploy instructions
+
+As this is enabled by default in the EKS monitoring module, visit [this example’s instructions](https://aws-observability.github.io/terraform-aws-observability-accelerator/eks/#prerequisites) which will provide the ADOT collector health dashboard after deployment
+
+## Disable ADOT health monitoring
+
+You can disable ADOT collector health metrics by setting the [variable](https://github.com/aws-observability/terraform-aws-observability-accelerator/blob/main/modules/eks-monitoring/variables.tf) enable_adotcollector_metrics to false.
+
+```
+variable "enable_adotcollector_metrics" {
+  description = "Enables collection of ADOT collector metrics"
+  type        = bool
+  default     = true
+}
+```

mkdocs.yml

@@ -35,6 +35,8 @@ nav:
       - Viewing logs: eks/logs.md
       - Tracing: eks/tracing.md
       - Teardown: eks/destroy.md
+  - AWS Distro for OpenTelemetry (ADOT):
+      - Monitoring ADOT collector health: adothealth/index.md
   - Amazon CloudWatch Container Insights:
       - Amazon EKS: container-insights/eks.md
   - Monitoring Managed Service for Prometheus Workspaces: workloads/managed-prometheus.md