You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are two ways to deploy the code in this directory.
3
+
This repository contains infrastructure deployment scripts for testing SDK example code. The infrastructure is managed through AWS CDK and can be deployed in two ways:
4
+
1.[deploy.py](#1-using-the-deploy-script)
5
+
2.[invoking the CDK directly](#2-invoking-cdk-directly)
4
6
5
-
## 1. Using the deploy script
7
+
## Option 1. Using `deploy.py`
6
8
7
-
To deploy any stack in this directory, run the [deploy.py](stacks/deploy.py) script.
9
+
The [deploy.py](stacks/deploy.py) script is the primary method for deploying the infrastructure stacks.
10
+
It exists in order to facilitate stack deployments to an infinite number of AWS accounts, without requiring the user to
11
+
fetch new tokens and set new variables for each deployment.
12
+
13
+
### Deployment types
14
+
The script handles three types of deployments:
15
+
16
+
1.**Images Stack** (`images`):
17
+
- Creates empty ECR private repositories for all tools listed in [targets.yaml](stacks/config/targets.yaml)
18
+
- Users must implement their own image versioning and pushing mechanism
19
+
- Example: GitHub Actions with OIDC provider works well for this purpose
20
+
21
+
2.**Admin Stack** (`admin`):
22
+
- Deploys event emission infrastructure
23
+
- Creates IAM policies for cross-account event subscription
24
+
-**Required**: Must be deployed before any plugin stacks
25
+
- Works with single or multiple accounts listed in [targets.yaml](stacks/config/targets.yaml)
26
+
27
+
3.**Plugin Stack** (`plugin`):
28
+
- Deploys two stacks to each account in [targets.yaml](stacks/config/targets.yaml):
29
+
1. Plugin stack that subscribes to admin stack events
30
+
2. Account nuker stack that cleans up residual test resources
31
+
- Requires `admin` stack to be deployed first
32
+
33
+
### Environment
34
+
It is designed to run from the command line interface (CLI) on macOS or Linux systems. You can use the default terminal emulator on macOS, such as zsh or bash, or any other terminal emulator of your choice.
35
+
36
+
### Why subprocess?
37
+
The script uses Python's subprocess module to execute the AWS Cloud Development Kit (CDK) command-line interface (CLI) commands. While the CDK provides a Python CDK library, 1) we use the TypeScript version per team standard, and 2) that Python CDK library does not expose a way to invoke the script itself from within a Python script. As a consequence, we are stuck using the `subprocess` module to invoke the CDK CLI commands for our TypeScript stack.
8
38
9
39
### Script Prerequisites
10
40
41
+
- Command line interface (CLI) on macOS installed, such as zsh or bash
11
42
- Python 3.11 installed
12
-
-`ada` or equivalent CLI library for fetching AWS credentials.
43
+
- AWS CLI and CDK installed and configured (NodeJS 18+)
44
+
- Admin-like IAM permissions on the role assumed (`AdministratorAccess` will work for non-production test environments).
45
+
- Configuration files [resources.yaml](stacks/config/resources.yaml) and [targets.yaml](stacks/config/targets.yaml)
46
+
- Environment variables set for:
47
+
-`TOKEN_TOOL`: Path to credential management tool
48
+
-`TOKEN_PROVIDER`: Identity provider for AWS credentials
13
49
- Dependencies installed in Python virtual environment:
- AWS CLI and CDK installed and configured (NodeJS 18+)
20
-
- Permissions to execute AWS CDK and shell commands (`AdministratorAccess` will work for non-production test environments)
21
-
- Configuration files `resources.yaml` and `targets.yaml`, which exist in the [stacks/config](stacks/config) directory within the same directory as the script
54
+
### Note on EnvVars `TOKEN_TOOL`and `TOKEN_PROVIDER`
55
+
These environment variables are designed to partly obscure the tooling used by AWS.
56
+
The `get_tokens` function on [deploy.py#L167](stacks/deploy.py#L167) may require additional refactoring to comply with whatever token tool you are using.
22
57
23
58
### Usage
24
59
25
60
#### Command Syntax
26
61
27
62
```bash
28
-
cd stacks ; python deploy.py <stack>
63
+
cd stacks ; python deploy.py <stack-type>
29
64
```
30
65
31
-
Replace `<stack>` with one of the supported stacks:
66
+
Replace `<stack-type>` with one of the supported stacks:
32
67
33
68
-`admin`: Deploys admin-specific resources.
34
69
-`images`: Deploys image-related resources.
35
70
-`plugin`: Deploys plugin-specific resources.
36
-
- To deploy only a specific language's plugin, pass `--language <language>` where language is an account in [targets.yaml](stacks/config/targets.yaml).
37
-
38
-
#### Additional Notes
39
-
40
-
The script automatically navigates to the required directory based on the type and language of deployment (typescript is the default).
41
-
42
-
Environment variables are set and used during the deployment process.
71
+
- To deploy only a specific language's plugin only, pass `--language <language>` where `<language>` is an account name in [targets.yaml](stacks/config/targets.yaml). E.g. `python`
43
72
44
-
Errors during command execution are caught and displayed.
45
-
46
-
The script includes a sleep period after deployment to avoid conflicts with simultaneous CDK operations.
47
-
48
-
Make sure to check the script's output for any errors or confirmation messages that indicate the deployment's success or failure. Adjust the config files as necessary to match your deployment requirements.
73
+
## Technical Notes
74
+
This creates some brittleness but provides necessary flexibility for cross-account deployments
49
75
76
+
#### Additional Notes
77
+
Some non-obvious quirks of the script include:
78
+
- programmatic file traversing to the required CDK directory based on the type and language of CDK deployment (`typescript` is the default).
79
+
- a random-seeming sleep period after deployment to avoid conflicts with the previous CDK operation that may have not killed its thread yet.
80
+
- more generally, extensive use of the `subprocess` module which creates some acceptable brittleness that may result in future regression.
50
81
---
51
82
52
-
## 2. Invoking CDK directly
53
-
54
-
The second option involves navigating to each stack directory and running the CDK commands.
55
-
56
-
The following instructions assume a "plugin account" (the AWS account where testing activities will occur) of "python" (corresponding to a Docker image) per [this repository's configuration](config/targets.yaml).
57
-
You can replace Python with any of the other languages listed in this repository's configuration.
58
-
59
-
To request an alternate configuration for your own repository or use case, please [submit an issue](https://github.com/awsdocs/aws-doc-sdk-examples/issues/new?labels=type%2Fenhancement&labels=Tools&title=%5BEnhancement%5D%3A+Weathertop+Customization+Request&&) with the `Tools` label.
83
+
## Option 2. Invoking CDK directly
60
84
61
-
### 1. Deploy Plugin Stack for your language (e.g. Python)
85
+
This option involves navigating to each stack directory([images](stacks/images), [admin](stacks/admin), or [plugin](stacks/plugin)) and running the `cdk` commands explained below.
62
86
63
-
User will:
87
+
Required steps for all stack types:
88
+
1. Set Python virtualenv within [plugin directory](stacks/plugin/admin).
89
+
1. Get AWS account tokens for target account.
90
+
1. Run `cdk bootstrap` and `cdk deploy`.
64
91
65
-
1. Set Python virtualenv within [plugin directory](plugin/admin).
66
-
1.`export LANGUAGE_NAME=python`.
67
-
1. Get AWS account tokens for plugin account.
68
-
1.`cdk bootstrap` and `cdk deploy`.
92
+
### Special details for `plugin` type
93
+
For the `plugin` type, there are a few important details:
94
+
1. User must also run `export LANGUAGE_NAME=python` if your tool is `python`.
95
+
1. For the stack to begin accepting test events, you must set `status` to `enabled` for your tool (e.g. `python`) in [targets.yaml](stacks/config/targets.yaml) and redeploy the `admin` stack.
96
+
1. To manually trigger test runs, [submit a test job](#submit-test-job) in AWS Batch.
69
97
70
-
### 2. Enable Consumer Stack to receive event notifications
98
+
## Testing & Validation
99
+
Users can trigger test runs from within the AWS Console after deploying the `plugin` stack for their chosen tool.
71
100
72
-
User will:
101
+
### Submit test job
73
102
74
-
1. Set `status` to `enabled` in [targets.yaml](config/targets.yaml) for your language
75
-
1. Raise PR.
103
+
Users can trigger test runs from within the AWS Console after deploying the `plugin` stack for their chosen tool.
76
104
77
-
Admin will:
78
-
79
-
1. Approve and merge PR.
80
-
1. Set Python virtualenv within [admin directory](stacks/admin).
81
-
1. Get Admin account tokens.
82
-
1.`cdk bootstrap` and `cdk deploy`.
83
-
1. Request that user [submit a test job](#3-submit-test-job).
84
-
85
-
### 3. Submit test job
86
-
87
-
User will:
88
-
89
-
1. Log into console for Python account
105
+
Steps:
106
+
1. Log into console for tool AWS account (e.g. `python`)
90
107
1. Navigate to "Job Definitions".
91
108
92
109
1. Click "Submit Job".
@@ -97,10 +114,12 @@ User will:
97
114
98
115
1. Click "Create job".
99
116
100
-
1.[Validate results of test job](#3-optional-view-test-job-results)
101
-
102
-
### 3. Optional: View CloudWatch job results in Batch
103
-
104
-
1. Navigate to a job
105
-
1. When status is `SUCCEEDED` or `FAILED`, click "Logging" tab.
117
+
1.[Validate results of test job](#view-test-run-results)
118
+
119
+
### View test run results
120
+
1. Log into console for tool AWS account (e.g. `python`)
121
+
1. Click `Jobs` and select the only job queue.
122
+
2. Toggle `Load all jobs`.
123
+
1. View job details by clicking the hyperlinked value in the `Name` field.
124
+
2. When status is `SUCCEEDED` or `FAILED`, click "Logging" tab.
0 commit comments