Skip to content

Commit 8f07b19

Browse files
authored
Multi Account Mixed Observability Observability Accelerator (#110)
* initial commit * AMP only try * AMP try * AMP only try * AMP only * AMP try * AMP Try * AMP try * AMP try * AMP try * AMP try * AMP Try * AMP try * AMP try * AMP try * AMP try * AMP try * AMP try * AMP Try * AMP try * AMP try * AMP try * AMP try * AMP try * AMP * AMP * AMP only * prod2 * prod2 * prod2 * prod2 * prod2 * amg * amg * amg * amg * amg * amg * amg * AMG * AMG * AMG * AMG * AMG * AMG * AMG * AMG and PROD2 * removed ssh from argocd * just credentialsSecretName * added credentialsType * all-in * AMG only * no ssh-key * with github-ssh-key * separated for public and private * try * updated createArgoAddonConfig * with go on mon-account * update * edit * added grafanaOperatorArgoAddonConfig * removed grafanaOperatorArgoAddonConfig * adding flux * new flux * updated * new bootstrapvals * added GrafanaOperatorAddon * new changes * update * resolved * updated * update * update * update * update * added mon * update * update * update * updated * update * update * updated bootstrapvals * updated amgEndpointUrl * updates * updated * updated with context from cdk.json * updates * update * updates * path * try * update * upd * upd * added CW in bootstrapValues * app update * update * upd * update * upd * converted to SSM SecureString * added pipelineSource * updates * update * SSM step * updates * upd * update * GO * more * upd * update * update * UPD * updates * update * update * doc update * more updates * upd * updated * rolechange * new * updated * split pipeline * commit * added * correct sampleStage * with account * update * commented out * upd * updated to prod1 * upd * upd * update * added new * update * removed getAMPInfo * doc updates * doc updates * doc updates * doc update and clean up script * updated doc * updated pipeline.ts * cleanup script and doc update * updated pipeline.ts to single statement * pipeline module * updated multi-account-central-pipeline to multi-account-COA-pipeline removed amp-iam-setup updated doc * doc update, architecture diagram and code updates * cleanup script update * cleanup script update * cleanup script update * update * added cwAssumeRoleName * updated cleanup * md update * updated doc * updated doc * Incorporating review comments removed comments in grafana-operator-construct removed comments in lib/multi-acc-new-eks-mixed-observability-pattern/cloudwatch-monitoring.ts renamed CreateIAMRoleNestedStack.ts to create-iam-role.ts getSSMSecureString.ts to get-ssm-securestring.ts GrafanaOperatorConstruct.ts to grafana-operator-index.ts grafanaoperatorsecretaddon.ts to grafana-operator-secret-addon.ts scripts to helpers * removed ServicePrincipal which is not in use * removed trailing whitespaces * moved policystatements outside of pipeline.ts * updated doc * updated 'scripts' ref to 'helpers' * updated codebuild role * rolled back to old way of creating iam role * fix with pipeline.ts * WORKING VERSION * policydoc outside * all policydocs are outside pipeline.ts * updated doc and some cleanup at pipeline.ts * doc update * fixed undefined assumerole with monitoringEnv * updated argocd app sync * updated sample runme gif * doc update * updated git-ssh-key to pipeline-account * removed git-ssh-key. sample apps are in publi repo * moved github-token creation to script * added categories to doc * removed post-deploy * Revert "removed post-deploy" This reverts commit d238188. * Revert "removed post-deploy" This reverts commit d238188. * Revert "added categories to doc" This reverts commit 3622641. * updated categories * updated command for running argo cd admin ui * updated cleanup script * updated doc * updated to artifacts/argocd-apps ref * updated progress check command * updated argo apps to aws-observability repo * updated broken URLs * fixed broken URLs in doc * removed unused images * added pre-req for tools * main readme with pattern docs, mkdocs, GH fix * updated README and doc for mkdocs * main readme update * moved argocdaddonconfig * removed createArgoAddonConfig * retaining one doc * retaining one doc * renamed in mkdocs
1 parent 3740a50 commit 8f07b19

33 files changed

+1877
-70
lines changed

README.md

Lines changed: 72 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -20,75 +20,81 @@ We provide curated metrics, logs, traces collection, cloudwatch dashboard, alert
2020
The individual patterns can be found in the [`lib`](https://github.com/aws-observability/cdk-aws-observability-accelerator/tree/main/lib) directory. Most of the patterns are self-explanatory, for some more complex examples please use this guide and docs/patterns directory for more information.
2121

2222
## Usage
23+
2324
Before proceeding, make sure [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) is installed on your machine.
2425

25-
To use the eks-blueprints and patterns module, you must have [Node.js](https://nodejs.org/en/) and [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) installed. You will also use `make` and `brew` to simplify build and other common actions.
26+
To use the eks-blueprints and patterns module, you must have [Node.js](https://nodejs.org/en/) and [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) installed. You will also use `make` and `brew` to simplify build and other common actions.
2627

2728
### Ubuntu Setup
29+
2830
Follow the below steps to setup and leverage cdk-aws-observability-accelerator in your Ubuntu Linux machine.
2931

30-
1. **Update the package list**
32+
1. **Update the package list**
3133

32-
Update the package list to ensure you're installing the latest versions.
34+
Update the package list to ensure you're installing the latest versions.
3335

34-
```bash
35-
sudo apt update
36-
```
36+
```bash
37+
sudo apt update
38+
```
3739

3840
2. **Install make**
3941

40-
```bash
41-
sudo apt install make
42-
```
42+
```bash
43+
sudo apt install make
44+
```
4345

4446
3. **Install Node.js and npm**
4547

46-
Install Node.js and npm using the NodeSource binary distributions.
47-
```bash
48-
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - &&\
49-
sudo apt-get install -y nodejs
50-
```
51-
52-
Note: The Node.js package from NodeSource includes npm
48+
Install Node.js and npm using the NodeSource binary distributions.
49+
50+
```bash
51+
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - &&\
52+
sudo apt-get install -y nodejs
53+
```
54+
55+
Note: The Node.js package from NodeSource includes npm
5356

5457
4. **Verify Node.js and npm Installation**
5558

56-
Check the installed version of Node.js:
59+
Check the installed version of Node.js:
5760

58-
```bash
59-
node -v
60-
```
61+
```bash
62+
node -v
63+
```
6164

62-
The output should be `v20.x.x`.
65+
The output should be `v20.x.x`.
6366

64-
Check the installed version of npm:
67+
Check the installed version of npm:
6568

66-
```bash
67-
npm -v
68-
```
69+
```bash
70+
npm -v
71+
```
6972

70-
The output should be a version greater than `9.7.x`.
73+
The output should be a version greater than `9.7.x`.
7174

72-
If your npm version is not `9.7.x` or above, update npm with the following command:
75+
If your npm version is not `9.7.x` or above, update npm with the following command:
7376

74-
```bash
75-
sudo npm install -g npm@latest
76-
```
77+
```bash
78+
sudo npm install -g npm@latest
79+
```
7780

78-
Verify the installed version by running `npm -v`.
81+
Verify the installed version by running `npm -v`.
7982

8083
5. Install brew on ubuntu by following instructions as detailed in [docs.brew.sh](https://docs.brew.sh/Homebrew-on-Linux)
81-
```bash
82-
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
83-
```
84-
Add Homebrew to your PATH
85-
```
86-
test -d ~/.linuxbrew && eval "$(~/.linuxbrew/bin/brew shellenv)"
87-
test -d /home/linuxbrew/.linuxbrew && eval "$(/home/linuxbrew/.linux brew/bin/brew shellenv)"
88-
test -r ~/.bash_profile && echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.bash_profile
89-
echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.profile
90-
91-
```
84+
85+
```bash
86+
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
87+
```
88+
89+
Add Homebrew to your PATH
90+
91+
```sh
92+
test -d ~/.linuxbrew && eval "$(~/.linuxbrew/bin/brew shellenv)"
93+
test -d /home/linuxbrew/.linuxbrew && eval "$(/home/linuxbrew/.linux brew/bin/brew shellenv)"
94+
test -r ~/.bash_profile && echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.bash_profile
95+
echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.profile
96+
97+
```
9298

9399
Post completing the above, continue from Step: [Repo setup](#repo-setup)
94100

@@ -98,14 +104,14 @@ Follow the below steps to setup and leverage `cdk-aws-observability-accelerator`
98104

99105
1. Install `make` and `node` using brew
100106

101-
```
107+
```sh
102108
brew install make
103109
brew install node
104110
```
105111

106112
2. Install `npm`
107113

108-
```
114+
```sh
109115
sudo npm install -g n
110116
sudo n stable
111117
```
@@ -121,7 +127,7 @@ v20.3.1
121127

122128
Update (provided Node version manager is installed): `n stable`. May require `sudo`.
123129

124-
- NPM version must be 8.4 or above:
130+
- NPM version must be 8.4 or above:
125131

126132
```bash
127133
$ npm -v
@@ -130,67 +136,72 @@ $ npm -v
130136

131137
Updating npm: `sudo n stable` where stable can also be a specific version above 8.4. May require `sudo`.
132138

133-
134139
### Repo setup
140+
135141
1. Clone the `cdk-aws-observability-accelerator` repository
136142

137-
```
143+
```sh
138144
git clone https://github.com/aws-observability/cdk-aws-observability-accelerator.git
139-
```
145+
```
140146

141147
PS: If you are contributing to this repo, please make sure to fork the repo, add your changes and create a PR against it.
142148

143149
2. Once you have cloned the repo, you can open it using your favourite IDE and run the below commands to install the dependencies and build the existing patterns.
144150

145151
- Install project dependencies.
146152

147-
```
153+
```text
148154
make deps
149155
```
150156

151157
- To view patterns that are available to be deployed, execute the following:
152158

153-
```
159+
```sh
154160
make build
155161
```
156162

157163
- To list the existing CDK AWS Observability Accelerator Patterns
158164

159-
```
165+
```text
160166
make list
161167
```
162168

163169
Note: Some patterns have a hard dependency on AWS Secrets (for example GitHub access tokens). Initially you will see errors complaining about lack of the required secrets. It is normal. At the bottom, it will show the list of patterns which can be deployed, in case the pattern you are looking for is not available, it is due to the hard dependency which can be fixed by following the docs specific to those patterns.
164170

165-
```
171+
```ps1
166172
To work with patterns use:
167173
$ make pattern <pattern-name> <list | deploy | synth | destroy>
168174
Example:
169175
$ make pattern single-new-eks-opensource-observability deploy
170176
171177
Patterns:
172178
179+
single-new-eks-cluster
173180
existing-eks-mixed-observability
174-
existing-eks-opensource-observability
181+
single-new-eks-opensource-observability
175182
single-new-eks-awsnative-observability
176-
single-new-eks-cluster
183+
single-new-eks-gpu-opensource-observability
177184
single-new-eks-graviton-opensource-observability
178185
single-new-eks-mixed-observability
179-
single-new-eks-opensource-observability
186+
existing-eks-opensource-observability
187+
existing-eks-awsnative-observability
188+
multi-acc-new-eks-mixed-observability
180189
```
181190

182191
- Bootstrap your CDK environment.
183192

184-
```
193+
```sh
185194
npx cdk bootstrap
186195
```
187196

188197
- You can then deploy a specific pattern with the following:
189198

190-
```
199+
```sh
191200
make pattern single-new-eks-opensource-observability deploy
192201
```
193202

203+
- To access instructions for individual patterns check documentation in `docs/patterns` directory.
204+
194205
# Developer Flow
195206

196207
## Modifications
@@ -201,17 +212,18 @@ All files are compiled to the dist folder including `lib` and `bin` directories.
201212
make compile
202213
```
203214

204-
The `compile` command is optimized to build only modified files and is fast.
215+
The `compile` command is optimized to build only modified files and is fast.
205216

206217
## New Patterns
207218

208219
To create a new pattern, please follow these steps:
209220

210-
1. Under lib create a folder for your pattern, such as `<pattern-name>-pattern`. If you plan to create a set of patterns that represent a particular subdomain, e.g. `security` or `hardening`, please create an issue to discuss it first. If approved, you will be able to create a folder with your subdomain name and group your pattern constructs under it.
211-
2. Blueprints generally don't require a specific class, however we use a convention of wrapping each pattern in a plain class like `<Pattern-Name>Pattern`. This class is generally placed in `index.ts` under your pattern folder.
221+
1. Under lib create a folder for your pattern, such as `<pattern-name>-pattern`. If you plan to create a set of patterns that represent a particular subdomain, e.g. `security` or `hardening`, please create an issue to discuss it first. If approved, you will be able to create a folder with your subdomain name and group your pattern constructs under it.
222+
2. Blueprints generally don't require a specific class, however we use a convention of wrapping each pattern in a plain class like `<Pattern-Name>Pattern`. This class is generally placed in `index.ts` under your pattern folder.
212223
3. Once the pattern implementation is ready, you need to include it in the list of the patterns by creating a file `bin/<pattern-name>.ts`. The implementation of this file is very light, and it is done to allow patterns to run independently.
213224

214225
Example simple synchronous pattern:
226+
215227
```typescript
216228
import SingleNewEksOpenSourceobservabilityPattern from '../lib/single-new-eks-opensource-observability-pattern';
217229
import { configureApp } from '../lib/common/construct-utils';
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
2+
import { configureApp, errorHandler } from '../lib/common/construct-utils';
3+
import { PipelineMultiEnvMonitoring } from '../lib/multi-acc-new-eks-mixed-observability-pattern';
4+
5+
const app = configureApp();
6+
7+
/*
8+
CDK Observability Accelerator pattern for Multiple EKS clusters in multiple AWS accounts
9+
*/
10+
11+
new PipelineMultiEnvMonitoring()
12+
.buildAsync(app)
13+
.catch((e) => {
14+
errorHandler(app, "Multi Account Multi EKS CDK Observability pattern setup errored. Ensure SSM SecureString Parameter /cdk-accelerator/cdk-context exists in COA_PIPELINE_REGION of pipeline account. See documentation for details.", e);
15+
});
409 KB
Loading
188 KB
Loading
224 KB
Loading
2.84 MB
Loading
12.2 MB
Loading

0 commit comments

Comments
 (0)