Skip to content

Commit 0f60fb8

Browse files
lewinkedrsbonclay7
lewinkedrs
and
bonclay7
authored
Docs (#69)
* working first run * removing core module dependencies * adding CW datasource * alarms MVP * readmes * Adding Screenshot * Adding billing note * adding billing module * Revert "adding billing module" This reverts commit 40d667e37db1036cd71a471ef2fde83ec02aaa13. reverting * adding billing module * Updating Screenshot * resolving feedback * removing unused modules * fmt * Support for tf 1.3.x * removing unused variables * support alarms for multiple workspaces * Updating Readme * docs first draft * indigo nav fix * Simplify docs * RUM White Logo * amp docs * removing billing docs * Change docs structure, reword infrastructure monitoring doc * Pre-commit fixes * drop dead code * Add concepts page * Update contributors * Update java * Update docs * Docs for workloads * Update docs site * typos * pre-commit fixes * Update pre-commit * Update pre-commit Co-authored-by: Rodrigue Koffi <[email protected]>
1 parent db0cb5d commit 0f60fb8

File tree

14 files changed

+1165
-14
lines changed

14 files changed

+1165
-14
lines changed

‎docs/concepts.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Concepts
2+
3+
## Prerequisites
4+
5+
All examples in this repository require the following tools installed
6+
7+
1. [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)
8+
2. [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
9+
3. [Kubectl](https://Kubernetes.io/docs/tasks/tools/)
10+
11+
### Minimum IAM Policy
12+
13+
To run the examples, you need a set of AWS IAM permissions. You can find an example of minimum
14+
permissions required [in this file](https://github.com/aws-observability/terraform-aws-observability-accelerator/blob/docs/docs/iam/min-iam-policy.json).
15+
16+
> **Note**: The policy resource is set as `*` to allow all resources, this is not a recommended practice.
17+
You should restrict instead to the ARNs when applicable.
18+
19+
### Terraform states and variables
20+
21+
By default, our examples are using local Terraform states. If you need
22+
your Terraform states to be saved remotely, on Amazon S3, visit the
23+
[terraform remote states](https://www.terraform.io/language/state/remote) documentation.
24+
25+
For simplicity, we use Terraform supported environment variables.
26+
You can also edit the `terraform.tfvars` files directly and deploy
27+
with `terraform apply -var-file=terraform.tfvars`. Terraform tfvars file can be useful if
28+
you need to track changes as part of a Git repository or CI/CD pipeline.
29+
30+
> **Note:** When using `tfvars` files, always be careful to not store and commit any secrets (keys, passwords, ...)
31+
32+
## Base module
33+
34+
The base module allows you to configure the AWS Observability services for your cluster and the AWS Distro for OpenTelemetry (ADOT) Operator as the signals collection mechanism.
35+
36+
Here is the minimum configuration to have a new Managed Grafana Workspace, Amazon Managed Service for Prometheus Workspace, ADOT Operator deployed for you and ready to receive your data.
37+
38+
```hcl
39+
module "eks_observability_accelerator" {
40+
source = "aws-observability/terraform-aws-observability-accelerator"
41+
aws_region = "eu-west-1"
42+
eks_cluster_id = "my-eks-cluster"
43+
}
44+
```
45+
46+
You can optionally reuse existing Workspaces to dissociate their lifecycle from the
47+
Terraform state.
48+
49+
```hcl
50+
module "eks_observability_accelerator" {
51+
source = "aws-observability/terraform-aws-observability-accelerator"
52+
aws_region = "eu-west-1"
53+
eks_cluster_id = "my-eks-cluster"
54+
55+
# prevents creation of a new Amazon Managed Prometheus workspace
56+
enable_managed_prometheus = false
57+
58+
# reusing existing Amazon Managed Prometheus Workspace
59+
managed_prometheus_workspace_id = "ws-abcd123..."
60+
61+
# prevents creation of a new Amazon Managed Grafana workspace
62+
enable_managed_grafana = false
63+
64+
managed_grafana_workspace_id = "g-abcdef123"
65+
grafana_api_key = var.grafana_api_key
66+
}
67+
```
68+
69+
View all the configuration options in the [module's documentation](https://github.com/aws-observability/terraform-aws-observability-accelerator#requirements)
70+
71+
## Workload modules
72+
73+
Workloads modules are focused Terraform modules provided in this repository. They essentially provide curated metrics collection, alerts and Grafana dashboards according to the use case. Most of those modules require the base module.
74+
75+
You can check the full workload modules list and their documentation [here](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/modules/workloads).
76+
77+
All the modules come with end-to-end deployable examples.
78+
79+
## Examples
80+
81+
[Examples](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/examples) put modules together in a ready to deploy terraform configuration as a starting point. With little to no configuration, you can run `terraform apply` and use the deployed resources on your AWS Account.
82+
83+
You can find **workload** examples like [Amazon EKS infrstructure monitoring](/terraform-aws-observability-accelerator/workloads/eks/) or [monitoring your Amazon Managed Service for Prometheus workspace](terraform-aws-observability-accelerator/workloads/managed-prometheus/) and more.
84+
85+
86+
## Getting started with AWS Observability services
87+
88+
If you are new to AWS Observability services, or want to dive deeper into them, check our [One Observability Workshop](https://catalog.workshops.aws/observability/) for a hands-on experience in a self-paced environement or at an AWS venue.

‎docs/contributors.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Contributors
2+
3+
The content on this site is maintained by the Solutions Architects from the
4+
AWS observability team with support from the AWS service teams and other
5+
volunteers from across the organization.
6+
7+
Our goal is to make it easier to use AWS Open Source Observability Services.
8+
9+
The core team include the following people:
10+
11+
* Abhi Khanna
12+
* Imaya Kumar Jagannathan
13+
* Jerome DECQ
14+
* Kevin Lewin
15+
* Michael Hausenblas
16+
* Munish Dabra
17+
* Ramesh Kumar Venkatraman
18+
* Rodrigue Koffi
19+
* Toshal Dudhwhala
20+
* Vara Bonthu
21+
* Vikram Venkataraman
22+
23+
We welcome the wider open source community and thank [those who contribute](https://github.com/aws-observability/terraform-aws-observability-accelerator/graphs/contributors)
24+
to this project.
25+
26+
Note that all information published on this site is available via the
27+
Apache 2.0 license.

‎docs/eks.md

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
# Amazon EKS cluster monitoring
2+
3+
This example demonstrates how to monitor your Amazon Elastic Kubernetes Service
4+
(Amazon EKS) cluster with the Observability Accelerator's EKS
5+
[infrastructure module](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/modules/workloads/infra).
6+
7+
Monitoring Amazon Elastic Kubernetes Service (Amazon EKS) has two categories:
8+
the control plane and the Amazon EKS nodes (with Kubernetes objects).
9+
The Amazon EKS control plane consists of control plane nodes that run the Kubernetes software,
10+
such as etcd and the Kubernetes API server. To read more on the components of an Amazon EKS cluster,
11+
please read the [service documentation](https://docs.aws.amazon.com/eks/latest/userguide/clusters.html).
12+
13+
The Amazon EKS infrastructure Terraform modules focuses on metrics collection to Amazon
14+
Managed Service for Prometheus using the [AWS Distro for OpenTelemetry Operator](https://docs.aws.amazon.com/eks/latest/userguide/opentelemetry.html) for Amazon EKS.
15+
Additionally, it provides default dashboards to get a comprehensible visibility on the nodes,
16+
namespaces, pods, and kubelet operations health. Finally, you get curated Prometheus recording rules
17+
and alerts to operate your cluster.
18+
19+
## Prerequisites
20+
21+
Make sure to complete the [prerequisites section](/terraform-aws-observability-accelerator/concepts/#prerequisites)
22+
before proceeding.
23+
24+
## Setup
25+
26+
### 1. Download sources and initialize Terraform
27+
28+
```
29+
git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
30+
cd examples/existing-cluster-with-base-and-infra
31+
terraform init
32+
```
33+
34+
### 2. AWS Region
35+
36+
Specify the AWS Region where the resources will be deployed:
37+
38+
```bash
39+
export TF_VAR_aws_region=xxx
40+
```
41+
42+
### 3. Amazon EKS Cluster
43+
44+
To run this example, you need to provide your EKS cluster name. If you don't
45+
have a cluster ready, visit [this example](/terraform-aws-observability-accelerator/helpers/new-eks-cluster.md)
46+
first to create a new one.
47+
48+
Specify your cluster name:
49+
50+
```bash
51+
export TF_VAR_eks_cluster_id=xxx
52+
```
53+
54+
### 4. Amazon Managed Service for Prometheus workspace (optional)
55+
56+
By default, we create an Amazon Managed Service for Prometheus workspace for you.
57+
However, if you have an existing workspace you want to reuse, edit and run:
58+
59+
```bash
60+
export TF_VAR_managed_prometheus_workspace_id=ws-xxx
61+
```
62+
63+
To create a workspace outside of Terraform's state, simply run:
64+
65+
```bash
66+
aws amp create-workspace --alias observability-accelerator --query '.workspaceId' --output text
67+
```
68+
69+
### 5. Amazon Managed Grafana workspace
70+
71+
To run this example you need an Amazon Managed Grafana workspace. If you have an existing workspace, edit and run:
72+
73+
```bash
74+
export TF_VAR_managed_grafana_workspace_id=g-xxx
75+
```
76+
77+
To create a new one, within this example's Terraform state (sharing the same lifecycle with all the
78+
other resources created by Terraform):
79+
80+
- Edit main.tf and set `enable_managed_grafana = true`
81+
- Run
82+
83+
```bash
84+
terraform init
85+
terraform apply -target "module.eks_observability_accelerator.module.managed_grafana[0].aws_grafana_workspace.this[0]"
86+
export TF_VAR_managed_grafana_workspace_id=$(terraform output --raw managed_grafana_workspace_id)
87+
```
88+
89+
### 6. Grafana API Key
90+
91+
Amazon Managed Grafana provides a control plane API for generating Grafana API keys.
92+
As a security best practice, we will provide to Terraform a short lived API key to
93+
run the `apply` or `destroy` command.
94+
95+
Ensure you have necessary IAM permissions (`CreateWorkspaceApiKey, DeleteWorkspaceApiKey`)
96+
97+
```bash
98+
export TF_VAR_grafana_api_key=`aws grafana create-workspace-api-key --key-name "observability-accelerator-$(date +%s)" --key-role ADMIN --seconds-to-live 1200 --workspace-id $TF_VAR_managed_grafana_workspace_id --query key --output text`
99+
```
100+
101+
## Deploy
102+
103+
Simply run this command to deploy the example
104+
105+
```bash
106+
terraform apply
107+
```
108+
109+
## Visualization
110+
111+
1. Prometheus datasource on Grafana
112+
113+
Open your Grafana workspace and under Configuration -> Data sources, you should see `aws-observability-accelerator`. Open and click `Save & test`. You should see a notification confirming that the Amazon Managed Service for Prometheus workspace is ready to be used on Grafana.
114+
115+
2. Grafana dashboards
116+
117+
Go to the Dashboards panel of your Grafana workspace. You should see a list of dashboards under the `Observability Accelerator Dashboards`
118+
119+
<img width="1540" alt="image" src="https://user-images.githubusercontent.com/10175027/190000716-29e16698-7c90-49d6-8c37-79ca1790e2cc.png">
120+
121+
Open a specific dashboard and you should be able to view its visualization
122+
123+
<img width="2056" alt="cluster headlines" src="https://user-images.githubusercontent.com/10175027/199110753-9bc7a9b7-1b45-4598-89d3-32980154080e.png">
124+
125+
2. Amazon Managed Service for Prometheus rules and alerts
126+
127+
Open the Amazon Managed Service for Prometheus console and view the details of your workspace. Under the `Rules management` tab, you should find new rules deployed.
128+
129+
<img width="1629" alt="image" src="https://user-images.githubusercontent.com/10175027/189301297-4865e75d-2d71-434f-b5d0-9750b3533632.png">
130+
131+
132+
To setup your alert receiver, with Amazon SNS, follow [this documentation](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-alertmanager-receiver.html)
133+
134+
135+
## Destroy resources
136+
137+
If you leave this stack running, you will continue to incur charges. To remove all resources
138+
created by Terraform, [refresh your Grafana API key](#6-grafana-api-key) and run the command below.
139+
140+
Be careful, this command will removing everything created by Terraform. If you wish
141+
to keep your Amazon Managed Grafana or Amazon Managed Service for Prometheus workspaces. Remove them
142+
from your terraform state before running the destroy command.
143+
144+
```bash
145+
terraform destroy
146+
```
147+
148+
To remove resources from your Terraform state, run
149+
150+
```bash
151+
# grafana workspace
152+
terraform state rm "module.eks_observability_accelerator.module.managed_grafana[0].aws_grafana_workspace.this[0]"
153+
154+
# prometheus workspace
155+
terraform state rm "module.eks_observability_accelerator.aws_prometheus_workspace.this[0]"
156+
```
157+
158+
159+
> **Note:** To view all the features proposed by this module, visit the [module documentation](https://github.com/aws-observability/terraform-aws-observability-accelerator/tree/main/modules/workloads/infra).

‎docs/helpers/new-eks-cluster.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Creating a new Amazon EKS cluster with VPC
2+
3+
> Note: This example is a subset from [this EKS Blueprint example](https://github.com/aws-ia/terraform-aws-eks-blueprints/tree/main/examples/eks-cluster-with-new-vpc)
4+
5+
This example deploys the following:
6+
7+
- New sample VPC, 3 Private Subnets and 3 Public Subnets
8+
- Internet gateway for Public Subnets and NAT Gateway for Private Subnets
9+
- EKS Cluster Control plane with one managed node group
10+
11+
## Prerequisites
12+
13+
Make sure to complete the [prerequisites section](/terraform-aws-observability-accelerator/concepts/#prerequisites)
14+
before proceeding.
15+
16+
17+
## Setup
18+
19+
### 1. Download sources and initialize Terraform
20+
21+
```
22+
git clone https://github.com/aws-observability/terraform-aws-observability-accelerator.git
23+
cd examples/eks-cluster-with-vpc/
24+
terraform init
25+
```
26+
27+
### 2. AWS Region
28+
29+
Specify the AWS Region where the resources will be deployed:
30+
31+
```bash
32+
export TF_VAR_aws_region=xxx
33+
```
34+
35+
## Deploy
36+
37+
Simply run this command to deploy the example
38+
39+
```bash
40+
terraform apply
41+
```
42+
43+
## Login to your cluster
44+
45+
EKS Cluster details can be extracted from terraform output or from AWS Console to get the name of cluster.
46+
Use the following commands in your local machine where you want to interact with your EKS Cluster.
47+
48+
### 1. Run `update-kubeconfig` command
49+
50+
`~/.kube/config` file gets updated with cluster details and certificate from the below command
51+
52+
aws eks --region <enter-your-region> update-kubeconfig --name <cluster-name>
53+
54+
### 2. List all the worker nodes by running the command below
55+
56+
kubectl get nodes
57+
58+
### 3. List all the pods running in `kube-system` namespace
59+
60+
kubectl get pods -n kube-system
61+
62+
## Cleanup
63+
64+
To clean up your environment, destroy the Terraform modules in reverse order.
65+
66+
Destroy the Kubernetes Add-ons, EKS cluster with Node groups and VPC
67+
68+
```sh
69+
terraform destroy -target="module.eks_blueprints_kubernetes_addons" -auto-approve
70+
terraform destroy -target="module.eks_blueprints" -auto-approve
71+
terraform destroy -target="module.vpc" -auto-approve
72+
```
73+
74+
Finally, destroy any additional resources that are not in the above modules
75+
76+
```sh
77+
terraform destroy -auto-approve
78+
```

0 commit comments

Comments
 (0)