Skip to content

Commit bb4744a

Browse files
ran-isenbergRan Isenberg
ran-isenberg
and
Ran Isenberg
authored
feature: use cookbook v4.2 (#20)
Co-authored-by: Ran Isenberg <ran.isenberg@ranthebuilder.cloud>
1 parent e4c91b4 commit bb4744a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

74 files changed

+1524
-587
lines changed

‎README.md‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,18 +71,25 @@ The documentation provides information about CDK deployment, makefile commands,
7171
![design](https://github.com/ran-isenberg/cookiecutter-serverless-python/blob/main/media/design.png?raw=true)
7272
<br></br>
7373

74+
75+
#### **Monitoring Design**
76+
![monitoring_design](https://github.com/ran-isenberg/aws-lambda-handler-cookbook/blob/main/docs/media/monitoring_design.png?raw=true)
77+
<br></br>
78+
7479
### **Features**
7580

7681
- Python Serverless service with a recommended file structure.
7782
- CDK infrastructure with infrastructure tests and security tests.
7883
- CI/CD pipelines based on Github actions that deploys to AWS with python linters, complexity checks and style formatters.
84+
- CI/CD pipeline deploys to dev/staging and production environment with different gates between each environment
7985
- Makefile for simple developer experience.
8086
- The AWS Lambda handler embodies Serverless best practices and has all the bells and whistles for a proper production ready handler.
81-
- AWS Lambda handler uses [AWS Lambda Powertools](https://awslabs.github.io/aws-lambda-powertools-python/).
87+
- AWS Lambda handler uses [AWS Lambda Powertools](https://docs.powertools.aws.dev/lambda-python/).
8288
- AWS Lambda handler 3 layer architecture: handler layer, logic layer and data access layer
8389
- Features flags and configuration based on AWS AppConfig
8490
- Idempotent API
85-
- Unit, infrastructure, security, integration and E2E tests.
91+
- CloudWatch dashboards - High level and low level including CloudWatch alarms
92+
- Unit, infrastructure, security, integration and end to end tests.
8693
<br></br>
8794

8895
## CDK Deployment
@@ -107,7 +114,8 @@ The utilities cover multiple aspect of a production-ready service, including:
107114
- [Dynamic Configuration & feature flags](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-part-6-feature-flags-configuration-best-practices)
108115
- [Start Your AWS Serverless Service With Two Clicks](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-part-7-how-to-use-the-aws-lambda-cookbook-github-template-project)
109116
- [CDK Best practices](https://github.com/ran-isenberg/aws-lambda-handler-cookbook)
110-
- [Idempotent API](https://www.ranthebuilder.cloud/post/serverless-api-idempotency-with-aws-lambda-powertools-and-cdk)
117+
- [Serverless Monitoring](https://www.ranthebuilder.cloud/post/how-to-effortlessly-monitor-serverless-applications-with-cloudwatch-part-one)
118+
- [API Idempotency](https://www.ranthebuilder.cloud/post/serverless-api-idempotency-with-aws-lambda-powertools-and-cdk)
111119

112120
<br></br>
113121
### Makefile Commands

‎hooks/post_gen_project.py‎

Lines changed: 2 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,10 @@
1-
import os
2-
from pathlib import Path
31
import subprocess
42

53
def main():
64
print("Running 'git init'")
75
subprocess.run(['git', 'init'], check=True)
8-
9-
print("Running 'pip install --upgrade pip pre-commit poetry'")
10-
subprocess.run(['pip', 'install', '--upgrade', 'pip', 'pre-commit', 'poetry'], check=True)
11-
12-
print("Running 'pre-commit install'")
13-
14-
subprocess.run(['pre-commit', 'install'], check=True)
15-
16-
print("Running 'poetry config --local virtualenvs.in-project true'")
17-
subprocess.run(['poetry', 'config', '--local', 'virtualenvs.in-project', 'true'], check=True)
18-
19-
print("Running 'poetry install'")
20-
subprocess.run(['poetry', 'install'], check=True)
21-
6+
print("Initializing project")
7+
subprocess.run(['make', 'dev'], check=True)
228
print("Project successfully initialized")
239
return
2410

‎media/monitoring_design.png‎

41.4 KB
Loading

‎{{cookiecutter.repo_name}}/.flake8‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[flake8]
2-
exclude = .eggs, setup.py, example, .cdk,out, .git, dist, *.md, *.yaml, *.txt, *.ini
2+
exclude = .eggs, setup.py, example, .cdk,out, .git, dist, *.md, *.yaml, *.txt, *.ini, build, cdk.json, cdk.context.json, cdk.out, node_modules
33
ignore = E203, E266, W503, BLK100, W291, I004
44
max-line-length = 150
55
max-complexity = 15

‎{{cookiecutter.repo_name}}/.github/FUNDING.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# These are supported funding model platforms
2+
3+
github: ran-isenberg
4+
patreon: # Replace with a single Patreon username
5+
open_collective: # Replace with a single Open Collective username
6+
ko_fi: # Replace with a single Ko-fi username
7+
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
8+
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
9+
liberapay: # Replace with a single Liberapay username
10+
issuehunt: # Replace with a single IssueHunt username
11+
otechie: # Replace with a single Otechie username
12+
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

‎{{cookiecutter.repo_name}}/.github/dependabot.yml‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,28 @@ updates:
33
- package-ecosystem: "github-actions"
44
directory: "/"
55
schedule:
6-
interval: "weekly"
6+
interval: "monthly"
77
commit-message:
88
prefix: chore
99
include: scope
1010

1111
- package-ecosystem: "pip"
1212
directory: "/"
1313
schedule:
14-
interval: "weekly"
14+
interval: "monthly"
1515
target-branch: "main"
1616
commit-message:
1717
prefix: chore
1818
include: scope
19+
20+
- package-ecosystem: "npm"
21+
directory: "/"
22+
schedule:
23+
interval: "weekly"
24+
target-branch: "develop"
25+
commit-message:
26+
prefix: chore
27+
include: scope
28+
allow:
29+
# Allow updates for AWS CDK
30+
- dependency-name: "aws-cdk"

{{cookiecutter.repo_name}}/.github/workflows/codeql.yml renamed to {{cookiecutter.repo_name}}/.github/workflows/codeql-analysis.yml.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ jobs:
4343

4444
steps:
4545
- name: Checkout repository
46-
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
46+
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
4747

4848
# Initializes the CodeQL tools for scanning.
4949
- name: Initialize CodeQL

‎{{cookiecutter.repo_name}}/.github/workflows/comment_issues.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,10 @@
11
name: Comment when opened
2+
3+
permissions:
4+
issues: write
5+
checks: read
6+
contents: read
7+
28
on:
39
issues:
410
types:

‎{{cookiecutter.repo_name}}/.github/workflows/main-serverless-service.yml‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# This workflow will install Python dependencies, run tests and lint with a single version of Python
2+
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
3+
name: Main Branch - Serverless Service CI/CD
4+
5+
permissions:
6+
contents: read
7+
8+
env:
9+
NODE_VERSION: "18"
10+
PYTHON_VERSION: "3.11"
11+
AWS_REGION: "us-east-1"
12+
13+
on:
14+
workflow_dispatch:
15+
16+
push:
17+
branches: [main]
18+
19+
jobs:
20+
staging:
21+
runs-on: ubuntu-latest
22+
environment: staging
23+
permissions:
24+
id-token: write # required for requesting the JWT (GitHub OIDC)
25+
steps:
26+
- run: |
27+
echo "🎉 The job was automatically triggered by a ${{ env.EVENT_NAME }} event." >> $GITHUB_STEP_SUMMARY
28+
echo "🐧 This job is now running on a ${{ env.OS_NAME }} ${{env.OS_ARCH}} server hosted by GitHub!" >> $GITHUB_STEP_SUMMARY
29+
echo "🔎 The name of your branch is ${{ env.BRANCH_NAME }} and your repository is ${{ env.REPO_NAME }}." >> $GITHUB_STEP_SUMMARY
30+
env:
31+
EVENT_NAME: ${{ github.event_name}}
32+
OS_NAME: ${{ runner.os }}
33+
OS_ARCH: ${{runner.arch }}
34+
BRANCH_NAME: ${{ github.ref }}
35+
REPO_NAME: ${{ github.repository }}
36+
- name: Check out repository code
37+
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
38+
- name: Install poetry
39+
run: pipx install poetry
40+
- name: Set up Python
41+
uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4.7.1
42+
with:
43+
python-version: ${{ env.PYTHON_VERSION }}
44+
cache: "poetry" # NOTE: poetry must be installed before this step, or else cache doesn't work
45+
- name: Set up Node
46+
uses: actions/setup-node@5e21ff4d9bc1a8cf6de233a3057d20ec6b3fb69d # v3.8.1
47+
with:
48+
node-version: ${{ env.NODE_VERSION }}
49+
cache: "npm"
50+
- name: Install dependencies
51+
run: make dev
52+
# NOTE: unit tests are connecting to AWS to instantiate boto3 clients/resources
53+
# once that's discussed we can move unit and infra tests as part of the fast quality standards
54+
# see https://github.com/ran-isenberg/serverless-python-demo/pull/38#discussion_r1299372169
55+
- name: Configure AWS credentials
56+
uses: aws-actions/configure-aws-credentials@010d0da01d0b5a38af31e9c3470dbfdabdecca3a # v4.0.1
57+
with:
58+
role-to-assume: ${{ secrets['AWS_ROLE'] }}
59+
role-session-name: ${{ env.SESSION_NAME }}
60+
aws-region: ${{ env.AWS_REGION }}
61+
env:
62+
SESSION_NAME: "github-${{github.sha}}-staging"
63+
- name: Deploy to AWS
64+
run: make deploy
65+
env:
66+
ENVIRONMENT: staging # Custom environment variable
67+
# NOTE: these run unit and integration tests
68+
# we can look into coverage collection only later to make it faster and less brittle (--collect-only)
69+
- name: Code coverage tests
70+
run: make coverage-tests
71+
env:
72+
ENVIRONMENT: staging # Custom environment variable
73+
- name: Codecov
74+
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # v3.1.4
75+
with:
76+
files: ./coverage.xml
77+
fail_ci_if_error: false # optional (default = false)
78+
verbose: false # optional (default = false)
79+
- name: Run E2E tests
80+
run: make e2e
81+
env:
82+
ENVIRONMENT: staging # Custom environment variable
83+
84+
production:
85+
runs-on: ubuntu-latest
86+
needs: [staging]
87+
environment: production
88+
permissions:
89+
id-token: write # required for requesting the JWT (GitHub OIDC)
90+
steps:
91+
- run: |
92+
echo "🎉 The job was automatically triggered by a ${{ env.EVENT_NAME }} event." >> $GITHUB_STEP_SUMMARY
93+
echo "🐧 This job is now running on a ${{ env.OS_NAME }} ${{env.OS_ARCH}} server hosted by GitHub!" >> $GITHUB_STEP_SUMMARY
94+
echo "🔎 The name of your branch is ${{ env.BRANCH_NAME }} and your repository is ${{ env.REPO_NAME }}." >> $GITHUB_STEP_SUMMARY
95+
env:
96+
EVENT_NAME: ${{ github.event_name}}
97+
OS_NAME: ${{ runner.os }}
98+
OS_ARCH: ${{runner.arch }}
99+
BRANCH_NAME: ${{ github.ref }}
100+
REPO_NAME: ${{ github.repository }}
101+
- name: Check out repository code
102+
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
103+
- name: Install poetry
104+
run: pipx install poetry
105+
- name: Set up Python
106+
uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4.7.1
107+
with:
108+
python-version: ${{ env.PYTHON_VERSION }}
109+
cache: "poetry" # NOTE: poetry must be installed before this step, or else cache doesn't work
110+
- name: Set up Node
111+
uses: actions/setup-node@5e21ff4d9bc1a8cf6de233a3057d20ec6b3fb69d # v3.8.1
112+
with:
113+
node-version: ${{ env.NODE_VERSION }}
114+
cache: "npm"
115+
- name: Install dependencies
116+
run: make dev
117+
# NOTE: unit tests are connecting to AWS to instantiate boto3 clients/resources
118+
# once that's discussed we can move unit and infra tests as part of the fast quality standards
119+
# see https://github.com/ran-isenberg/serverless-python-demo/pull/38#discussion_r1299372169
120+
- name: Configure AWS credentials
121+
uses: aws-actions/configure-aws-credentials@010d0da01d0b5a38af31e9c3470dbfdabdecca3a # v4.0.1
122+
with:
123+
role-to-assume: ${{ secrets['AWS_ROLE'] }}
124+
role-session-name: ${{ env.SESSION_NAME }}
125+
aws-region: ${{ env.AWS_REGION }}
126+
env:
127+
SESSION_NAME: "github-${{github.sha}}-production"
128+
- name: Deploy to AWS
129+
run: make deploy
130+
env:
131+
ENVIRONMENT: production # Custom environment variable
132+
133+
publish_github_pages:
134+
runs-on: ubuntu-latest
135+
needs: [production]
136+
permissions:
137+
contents: write # for docs push
138+
if: contains('refs/heads/main', github.ref)
139+
steps:
140+
- name: Check out repository code
141+
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
142+
- name: Set up Python
143+
uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4.7.1
144+
with:
145+
python-version: ${{ env.PYTHON_VERSION }}
146+
- name: Set up Node
147+
uses: actions/setup-node@5e21ff4d9bc1a8cf6de233a3057d20ec6b3fb69d # v3.8.1
148+
with:
149+
node-version: ${{ env.NODE_VERSION }}
150+
cache: "npm"
151+
- name: Install dependencies
152+
run: make dev
153+
- name: Generate docs
154+
run: |
155+
poetry run mkdocs gh-deploy --force

‎{{cookiecutter.repo_name}}/.github/workflows/pr-serverless-service.yml‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# This workflow will install Python dependencies, run tests and lint with a single version of Python
2+
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
3+
name: PR - Serverless Service CI/CD
4+
5+
permissions:
6+
contents: read
7+
8+
env:
9+
NODE_VERSION: "18"
10+
PYTHON_VERSION: "3.11"
11+
AWS_REGION: "us-east-1"
12+
13+
on:
14+
workflow_dispatch:
15+
16+
pull_request:
17+
branches: [main]
18+
19+
jobs:
20+
quality_standards:
21+
runs-on: ubuntu-latest
22+
steps:
23+
- run: |
24+
echo "🎉 The job was automatically triggered by a ${{ env.EVENT_NAME }} event." >> $GITHUB_STEP_SUMMARY
25+
echo "🐧 This job is now running on a ${{ env.OS_NAME }} ${{env.OS_ARCH}} server hosted by GitHub!" >> $GITHUB_STEP_SUMMARY
26+
echo "🔎 The name of your branch is ${{ env.BRANCH_NAME }} and your repository is ${{ env.REPO_NAME }}." >> $GITHUB_STEP_SUMMARY
27+
env:
28+
EVENT_NAME: ${{ github.event_name}}
29+
OS_NAME: ${{ runner.os }}
30+
OS_ARCH: ${{runner.arch }}
31+
BRANCH_NAME: ${{ github.ref }}
32+
REPO_NAME: ${{ github.repository }}
33+
- name: Check out repository code
34+
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
35+
- name: Install poetry
36+
run: pipx install poetry
37+
- name: Set up Python
38+
uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4.7.1
39+
with:
40+
python-version: ${{ env.PYTHON_VERSION }}
41+
cache: "poetry" # NOTE: poetry must be installed before this step, or else cache doesn't work
42+
- name: Install dependencies
43+
run: make dev
44+
- name: pre commit
45+
run: make pre-commit
46+
- name: Formatting and Linting
47+
run: make lint
48+
- name: Complexity scan
49+
run: make complex
50+
tests:
51+
needs: quality_standards
52+
runs-on: ubuntu-latest
53+
environment: dev
54+
permissions:
55+
id-token: write # required for requesting the JWT (GitHub OIDC)
56+
steps:
57+
- name: Check out repository code
58+
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
59+
- name: Install poetry
60+
run: pipx install poetry
61+
- name: Set up Python
62+
uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4.7.1
63+
with:
64+
python-version: ${{ env.PYTHON_VERSION }}
65+
cache: "poetry" # NOTE: poetry must be installed before this step, or else cache doesn't work
66+
- name: Set up Node
67+
uses: actions/setup-node@5e21ff4d9bc1a8cf6de233a3057d20ec6b3fb69d # v3.8.1
68+
with:
69+
node-version: ${{ env.NODE_VERSION }}
70+
cache: "npm"
71+
- name: Install dependencies
72+
run: make dev
73+
# NOTE: unit tests are connecting to AWS to instantiate boto3 clients/resources
74+
# once that's discussed we can move unit and infra tests as part of the fast quality standards
75+
# see https://github.com/ran-isenberg/serverless-python-demo/pull/38#discussion_r1299372169
76+
- name: Configure AWS credentials
77+
uses: aws-actions/configure-aws-credentials@010d0da01d0b5a38af31e9c3470dbfdabdecca3a # v4.0.1
78+
with:
79+
role-to-assume: ${{ secrets.AWS_ROLE }}
80+
role-session-name: ${{ env.SESSION_NAME }}
81+
aws-region: ${{ env.AWS_REGION }}
82+
env:
83+
SESSION_NAME: "github-${{github.sha}}-dev"
84+
- name: Unit tests
85+
run: make unit
86+
- name: Infrastructure tests
87+
run: make infra-tests
88+
- name: Deploy to AWS
89+
run: make deploy
90+
# NOTE: these run unit and integration tests
91+
# we can look into coverage collection only later to make it faster and less brittle (--collect-only)
92+
- name: Code coverage tests
93+
run: make coverage-tests
94+
- name: Codecov
95+
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # v3.1.4
96+
with:
97+
files: ./coverage.xml
98+
fail_ci_if_error: false # optional (default = false)
99+
verbose: false # optional (default = false)
100+
- name: Run E2E tests
101+
run: make e2e
102+
- name: Destroy stack
103+
if: always()
104+
run: make destroy

0 commit comments

Comments
 (0)