Merge pull request #278 from aniadev-aws/main

Add Getting Started on CloudWatch guide

Author: sguruvar

docusaurus/docs/guides/getting-started/configure-agents-collectors.md

@@ -0,0 +1,97 @@
+---
+sidebar_position: 3
+---
+
+# Configure Agents/Collectors
+
+Once you have your monitoring account structure in place, you will need to configure your applications, services, and other infrastructure components to send telemetry to CloudWatch.
+
+This is a high-level guide to configuring your agents and collectors. For in-depth guidance, please refer to various sections in this best practices guide.
+
+## Amazon EKS
+
+For EKS, the simplest way to configure observability it to use the Amazon EKS add-on.  This will install Container Insights with enhanced observability for Amazon EKS. The add-on installs the CloudWatch agent to send infrastructure metrics from the cluster, installs Fluent Bit to send container logs, and also enables CloudWatch Application Signals to send application performance telemetry. (It is configurable if you do not want Application Signals, Container Insights, etc.)
+
+Typically, the Amazon CloudWatch Observability EKS add-on is installed as a DaemonSet.
+
+Some options for EKS are:
+
+### CloudWatch Agent for EKS
+
+- Amazon CloudWatch Observability EKS add-on
+- Amazon CloudWatch Observability Helm Chart
+
+### OTEL Collector for EKS
+
+Alternatively, if you want to use the OTEL collector, you can:
+- Configure AWS Exporters
+- Set your OTLP exporter to point to the log and traces OTLP endpoints
+- Define your processing pipelines
+- Instrument your application using OTEL libraries (if required)
+
+## Amazon ECS
+
+For ECS, you can enable Container Insights to collect infrastructure metrics for your clusters. You can also deploy Application Signals to collect application performance telemetry and associated traces. For logs, you can use the awslogs driver to send your log data to CloudWatch, or you can use OpenTelemetry collectors to send the data.
+
+Some options for EKS are:
+
+### CloudWatch Agent for ECS
+
+- Enable Container Insights
+- Deploy Application Signals (optional)
+- Use the awslogs log driver
+
+### OTEL Collector for ECS
+
+Alternatively, you can:
+- Run as a sidecar
+- Configure AWS Exporters
+- Set OTLP endpoints
+- Define processing pipelines
+- Instrument applications (if required)
+
+## Amazon EC2 and On-Premises
+
+CloudWatch agent can be used to send telemetry data from EC2 instances, other virtual machines, and on-premises servers to CloudWatch.
+
+### Deployment Options
+
+- **Workload Detection for EC2** – Provides an automated way to deploy the agent
+
+![EC2 Workload Detection](../../images/GettingStarted/ec2workloaddetection.png)
+
+- **Systems Manager** – Deploy and configure the agent using AWS Systems Manager
+- **Custom Automation** – Use your own automation tools
+- **Manual Installation** – Install manually for specific use cases
+
+You can configure/customize telemetry via config file (automatically or manually), and there is a wizard available to help you fine-tune your settings.
+
+### OTEL Collector for EC2
+
+You can also use the OTEL collector with:
+
+**OTLP Exporters:**
+
+![OTLP Configuration](../../images/GettingStarted/otlp.png)
+
+Use OTLP exporters for trace and log OTLP endpoints.
+
+**AWS-Specific Exporters:**
+
+![ADOT Configuration](../../images/GettingStarted/adot.png)
+
+Use AWS-specific exporters and processing pipelines.
+
+## Summary
+
+To summarize:
+1. Choose the appropriate agent/collector for your compute platform (EKS, ECS, EC2)
+2. Deploy using automated methods (add-ons, Helm charts, Systems Manager) or manual installation
+3. Configure telemetry collection based on your requirements
+4. Optionally use OpenTelemetry for vendor-neutral instrumentation
+
+For detailed configuration guides, refer to the specific sections in this best practices guide for your compute platform and observability tools.
+
+## Next Steps
+
+Continue to [Dashboards and Alerts](./dashboards-alerts.md)

docusaurus/docs/guides/getting-started/dashboards-alerts.md

@@ -0,0 +1,54 @@
+---
+sidebar_position: 4
+---
+
+# Dashboards and Alerts
+
+Once your telemetry is flowing, you can set up dashboards and alerts relevant to your use case.
+
+## Curated Dashboards
+
+Be sure to leverage curated dashboards which you can find under various parts of the CloudWatch console.
+
+For example, you will find automated dashboards for many services (such as Lambda, EC2, API Gateway, and many more) under Dashboards.
+
+If you are leveraging Application Signals, you will find application maps and dashboards under Application Signals (APM). In addition, you will find uninstrumented services which will highlight any gaps in observability.
+
+## Custom Dashboards
+
+You will also need to design your own business-specific dashboards. Refer to this guide on how to design your dashboards for operational excellence: [Building Dashboards for Operational Visibility](https://aws.amazon.com/builders-library/building-dashboards-for-operational-visibility/)
+
+## CloudWatch Alarms
+
+You will also create alerts (or Alarms in CloudWatch) to signal any problems with your services and infrastructure. You can create alarms in your monitoring account for centralized alarm visibility or/and individual alarms in local accounts.
+
+### Alarm Recommendations
+
+If you are unsure how to get started, Alarm Recommendations will help you. Alarm recommendations are based on monitoring best practices. Review the recommended alarm configurations before creating an alarm.
+
+For more details, see [Alarm recommendations for AWS services](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Best_Practice_Recommended_Alarms_AWS_Services.html).
+
+## Service Level Objectives (SLOs)
+
+You can also create SLOs and associated alarms to help you track important KPIs.
+
+For more information, see [CloudWatch SLOs](../../tools/slos.md).
+
+## Summary
+
+This concludes the Getting Started on CloudWatch guide. Here are the steps we covered:
+
+1. **Setup Monitoring and Source Accounts** – Configured cross-account observability to centralize telemetry data from multiple AWS accounts and regions
+2. **Setup Unified Data Store** – Centralized log data into a single account and region for unified querying and analysis
+3. **Configure Agents/Collectors** – Deployed CloudWatch agents and/or OpenTelemetry collectors to send telemetry from your applications and infrastructure
+4. **Dashboards and Alerts** – Created dashboards for visibility and alarms to monitor the health of your services
+
+## Next Steps
+
+For more in-depth guidance on specific topics, refer to the detailed sections throughout this best practices guide:
+
+- [Containers (ECS/EKS)](../containers/aws-native/eks/amazon-cloudwatch-container-insights.md)
+- [Serverless](../serverless/aws-native/lambda-based-observability.md)
+- [Operational Guides](../operational/observability-driven-dev.md)
+- [Cost Optimization](../cost/cost-visualization/cost.md)
+- [Signal Collection](../signal-collection/emf.md)

docusaurus/docs/guides/getting-started/setup-monitoring-source-accounts.md

@@ -0,0 +1,63 @@
+---
+sidebar_position: 1
+---
+
+# Setup Monitoring and Source Accounts
+
+In the majority of cases, customers need to visualize and correlate telemetry data from multiple AWS accounts because their services run across many accounts, and sometimes, many regions.
+
+If you only plan to run your observability and services in a single account, you can skip this step.
+
+The first step is to setup your monitoring and source accounts and specify exactly what telemetry you wish to share. You will leverage cross-account observability to do so. Please note that this works on a per-region basis.
+
+For more detailed instructions on how to setup cross-account observability, please refer to the [CloudWatch Cross-Account Observability](../cloudwatch_cross_account_observability.md) guide.
+
+## Monitoring Account
+
+Nominate a monitoring account from which you want to view the telemetry data in a centralized manner.
+
+Then define which accounts will share the data with your monitoring account. You can choose all accounts in your AWS organization or pick individual source accounts. You will also specify what telemetry data you want to share with the monitoring account (e.g. logs, metrics, traces, application signals etc).
+
+You will then [link the source accounts](../cloudwatch_cross_account_observability.md#step-2-link-source-accounts-to-the-monitoring-account) to complete the setup.
+
+Your typical monitoring account structure will look similar to this:
+
+![Monitoring Account Structure](../../images/GettingStarted/monitoring-acct-struct.png)
+
+You will [configure](../cloudwatch_cross_account_observability.md#step-1-set-up-a-monitoring-account) this on a per-region basis in CloudWatch settings.
+
+:::info
+With cross-account observability, logs and metrics are NOT copied from the source accounts, but trace data is copied to the monitoring account (with trace copy to the 1st monitoring account included at no additional cost). You simply view logs, metrics, traces and other telemetry centrally.
+:::
+
+## Multiple Monitoring Accounts
+
+Each monitoring account can be linked with up to 100,000 source accounts.
+
+However, there may be operational situations where you need multiple monitoring accounts. You can have multiple monitoring accounts based on your own requirements. This setup might look something like this:
+
+![Multiple Monitoring Accounts](../../images/GettingStarted/multiple-mon-accts.png)
+
+:::info
+If you need to share data from a single source account with multiple monitoring accounts, that is also configurable as each source account can share data with up to 5 monitoring accounts.
+:::
+
+## Telemetry Control
+
+You also have control of what telemetry data you share with the ability to specify metric and log filters allowing you extra granularity.
+
+![Telemetry Configuration](../../images/GettingStarted/telemetry-config.png)
+
+You will now be able to [visualize and query cross-account](../cloudwatch_cross_account_observability.md#querying-cross-account-telemetry-data) data from multiple accounts in a single monitoring account (per region).
+
+## Summary
+
+To summarize:
+1. Nominate and configure monitoring account(s)
+2. Configure source accounts
+3. Fine tune which telemetry you want to share
+4. Visualize and query all the source account data from the monitoring account
+
+## Next Steps
+
+Continue to [Setup Unified Data Store](./setup-unified-data-store.md)

docusaurus/docs/guides/getting-started/setup-unified-data-store.md

@@ -0,0 +1,69 @@
+---
+sidebar_position: 2
+---
+
+# Setup Unified Data Store
+
+In addition to cross-account observability, you can also centralize (copy) logs to a central region and a central account. The first copy is included at no additional cost!
+
+The CloudWatch Unified Data Store helps you build a unified repository at scale for your organization. You can collect, shape, and analyze your telemetry from here.
+
+## Overview
+
+CloudWatch Unified Data Store allows you to centralize your application, operational, security, and compliance data in one place. This means you can centralize your log data from multiple AWS accounts/regions, as well as from third-party tools into a single account and region for central querying and analysis.
+
+![Unified Data Store](../../images/GettingStarted/UDS.png)
+
+You can use the Unified Data Store in conjunction with cross-account observability or by itself.
+
+## Key Benefits
+
+- **Centralize all observability data** – Consolidate operational, security, and compliance data from AWS services and third-party sources into one unified store across multiple accounts and regions
+
+- **Eliminate data silos and duplication** – Remove unnecessary ETL pipelines, reduce storage costs, and simplify management by storing data once in a single location
+
+- **Accelerate troubleshooting and reduce MTTR** – Get faster insights with intuitive faceted queries, natural language search, and advanced visualizations through CloudWatch Logs Insights and Amazon OpenSearch Service
+
+- **Unlock business intelligence with flexible analytics** – Analyze unified data using your choice of tools including Amazon Athena, QuickSight, SageMaker, Apache Spark, and third-party Iceberg-compatible platforms without data duplication
+
+## Configuration Steps
+
+This is configured at the organization level:
+
+### Step 1: Configure Root Account
+
+In your root account:
+1. Turn on trusted access
+2. Identify a delegate account for the centralized datastore
+
+### Step 2: Configure Centralized Account
+
+In your centralized account, create centralization rule(s) including:
+- Organization ID or source accounts
+- Source regions
+
+## Centralization Rules
+
+You can decide and configure:
+
+1. **Source accounts** – Choose which source accounts you want to copy the data from and filter those by Organization, OU, or account ID
+2. **Source regions** – Select which source regions to copy the data from
+3. **Backup region** – Configure a backup region for a second copy of the data (optional)
+4. **Log group filters** – Filter log groups by name, prefix, or keyword using a number of operators
+5. **Multiple rules** – Configure multiple centralization rules based on your requirements
+
+Your account structure for centralized logging may look something like this:
+
+![Centralized Logs](../../images/GettingStarted/centralised-logs.png)
+
+## Summary
+
+To summarize:
+1. Enable trusted access in root account and identify delegate account
+2. Create centralization rules in the centralized account
+3. Configure source accounts, regions, and log group filters
+4. Optionally configure backup regions for redundancy
+
+## Next Steps
+
+Continue to [Configure Agents/Collectors](./configure-agents-collectors.md)