Skip to content

Commit 5808ba6

Browse files
abarciauskas-bgseabarciauskas-bgse
authored
Merge pull request #47 from developmentseed/feature/add-action-for-veda-deploy
Feature/add action for veda deploy
2 parents 3f8a385 + bb61ec0 commit 5808ba6

File tree

9 files changed

+115
-65
lines changed

9 files changed

+115
-65
lines changed

‎.github/actions/cdk-deploy/action.yml‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
name: Deploy
2+
3+
inputs:
4+
env_aws_secret_name:
5+
required: false
6+
type: string
7+
default: ''
8+
dir:
9+
required: false
10+
type: string
11+
default: "."
12+
script_path:
13+
required: true
14+
type: string
15+
default: ''
16+
skip_deploy:
17+
required: false
18+
type: string
19+
default: 'false'
20+
21+
runs:
22+
using: "composite"
23+
steps:
24+
- name: Install node and related deps
25+
uses: actions/setup-node@v4
26+
with:
27+
node-version: 20
28+
29+
- uses: actions/cache@v3
30+
with:
31+
path: ~/.npm
32+
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
33+
34+
- name: Install AWS CDK
35+
shell: bash
36+
run: npm install -g aws-cdk@2
37+
38+
- name: Install uv
39+
uses: astral-sh/setup-uv@v3
40+
with:
41+
version: "0.5.*"
42+
43+
- name: Set up Python
44+
uses: actions/setup-python@v5
45+
with:
46+
python-version: '3.12'
47+
48+
- name: Install dependencies
49+
shell: bash
50+
working-directory: ${{ inputs.dir }}
51+
run: |
52+
uv sync --only-group deployment
53+
uv run npm install
54+
uv pip install boto3 # for the generate_env_file.py script
55+
56+
- name: Get relevant environment configuration from aws secrets
57+
if: inputs.env_aws_secret_name != ''
58+
shell: bash
59+
env:
60+
AWS_DEFAULT_REGION: us-west-2
61+
run: python ${{ inputs.script_path }} --secret-id ${{ inputs.env_aws_secret_name }} --env-file ${{ inputs.dir }}/.env
62+
63+
- name: CDK Synth
64+
shell: bash
65+
working-directory: ${{ inputs.dir }}
66+
run: uv run --only-group deployment npm run cdk -- synth
67+
68+
- name: Check Asset Sizes
69+
shell: bash
70+
working-directory: ${{ inputs.dir }}
71+
run: |
72+
MAX_SIZE_BYTES=262144000 # 262 MB in bytes
73+
for dir in cdk.out/asset.*; do
74+
if [ -d "$dir" ]; then
75+
size=$(du -sb "$dir" | cut -f1)
76+
if [ "$size" -gt $MAX_SIZE_BYTES ]; then
77+
echo "Directory $dir exceeds 262 MB with size $size bytes (max: $MAX_SIZE_BYTES bytes)."
78+
exit 1
79+
fi
80+
echo "Asset directory $dir size: $size bytes"
81+
fi
82+
done
83+
echo "All asset directories are within size limits."
84+
85+
- name: Deploy Test
86+
if: ${{ inputs.skip_deploy == 'false' }}
87+
id: deploy_titiler_cmr_stack
88+
working-directory: ${{ inputs.dir }}
89+
run: uv run cdk deploy --all --require-approval never --outputs-file ${HOME}/cdk-outputs.json
90+
shell: bash
91+
env:
92+
TITILER_CMR_ADDITIONAL_ENV: '{"TITILER_CMR_S3_AUTH_STRATEGY":"iam"}'

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Test and Deploy
1+
name: Test
22

33
# Triggers on pushes to main, dev and tags.
44
on:
@@ -56,60 +56,3 @@ jobs:
5656
5757
- name: Run tests
5858
run: uv run pytest
59-
60-
deploy:
61-
needs: [tests]
62-
runs-on: ubuntu-latest
63-
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop' || startsWith(github.ref, 'refs/tags/v')
64-
65-
defaults:
66-
run:
67-
working-directory: infrastructure/aws
68-
69-
steps:
70-
- uses: actions/checkout@v4
71-
72-
- name: Configure AWS credentials
73-
uses: aws-actions/configure-aws-credentials@v2
74-
with:
75-
role-to-assume: ${{ secrets.deploy_role_arn }}
76-
role-session-name: samplerolesession
77-
aws-region: us-west-2
78-
79-
- name: Set up node
80-
uses: actions/setup-node@v4
81-
with:
82-
node-version: '14.x'
83-
84-
- name: Install cdk
85-
run: npm install -g
86-
87-
- name: Set up Python
88-
uses: actions/setup-python@v5
89-
with:
90-
python-version: '3.12'
91-
92-
- name: Install dependencies
93-
run: |
94-
python -m pip install --upgrade pip
95-
python -m pip install -r requirements-cdk.txt
96-
97-
# Build and deploy to the development environment whenever there is a push to main or dev
98-
- name: Build & Deploy Development
99-
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop'
100-
run: npm run cdk -- deploy titiler-cmr-staging --require-approval never
101-
env:
102-
# STACK_ALARM_EMAIL: ${{ secrets.ALARM_EMAIL }}
103-
STACK_ROLE_ARN: ${{ secrets.lambda_role_arn }}
104-
STACK_STAGE: staging
105-
STACK_ADDITIONAL_ENV: '{"TITILER_CMR_S3_AUTH_STRATEGY":"iam", "TITILER_CMR_API_DEBUG":"TRUE"}'
106-
107-
# Build and deploy to production deployment whenever there a new tag is pushed
108-
- name: Build & Deploy Production
109-
if: startsWith(github.ref, 'refs/tags/v')
110-
run: npm run cdk -- deploy titiler-cmr-production --require-approval never
111-
env:
112-
# STACK_ALARM_EMAIL: ${{ secrets.ALARM_EMAIL }}
113-
STACK_ROLE_ARN: ${{ secrets.lambda_role_arn }}
114-
STACK_STAGE: production
115-
STACK_ADDITIONAL_ENV: '{"TITILER_CMR_S3_AUTH_STRATEGY":"iam"}'

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,10 @@ TITILER_CMR_S3_AUTH_ACCESS=external uvicorn titiler.cmr.main:app --reload
9595

9696
The application will be available at this address: [http://localhost:8000/api.html](http://localhost:8000/api.html)
9797

98+
## Deployment to AWS
99+
100+
Deployment to AWS is currently triggered using [veda-deploy](https://github.com/NASA-IMPACT/veda-deploy). veda-deploy checks out this repo as a submodule and then executes [.github/actions/cdk-deploy/action.yml](.github/actions/cdk-deploy/action.yml) (see also: [veda-deploy/.github/workflows/deploy.yml](https://github.com/NASA-IMPACT/veda-deploy/blob/dev/.github/workflows/deploy.yml)). For more details, please review the [veda-deploy README section on adding new components](https://github.com/NASA-IMPACT/veda-deploy/tree/dev?tab=readme-ov-file#add-new-components).
101+
98102
## Contribution & Development
99103

100104
See [CONTRIBUTING.md](https://github.com/developmentseed/titiler-cmr/blob/develop/CONTRIBUTING.md)

‎infrastructure/aws/cdk/app.py‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,15 @@
33
import os
44
from typing import Any, Dict, List, Optional
55

6-
from aws_cdk import App, CfnOutput, Duration, Stack, Tags
7-
from aws_cdk import aws_apigatewayv2_alpha as apigw
6+
from aws_cdk import App, CfnOutput, Duration, Stack, Tags, aws_lambda
7+
from aws_cdk import aws_apigatewayv2 as apigw
88
from aws_cdk import aws_cloudwatch as cloudwatch
99
from aws_cdk import aws_cloudwatch_actions as cloudwatch_actions
1010
from aws_cdk import aws_iam as iam
11-
from aws_cdk import aws_lambda
1211
from aws_cdk import aws_logs as logs
1312
from aws_cdk import aws_sns as sns
1413
from aws_cdk import aws_sns_subscriptions as subscriptions
15-
from aws_cdk.aws_apigatewayv2_integrations_alpha import HttpLambdaIntegration
14+
from aws_cdk.aws_apigatewayv2_integrations import HttpLambdaIntegration
1615
from config import StackSettings
1716
from constructs import Construct
1817

‎infrastructure/aws/cdk/config.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,5 +37,5 @@ class StackSettings(BaseSettings):
3737
alarm_email: Optional[str] = None
3838

3939
model_config = SettingsConfigDict(
40-
env_prefix="STACK_", env_file=".env", extra="ignore"
40+
env_prefix="TITILER_CMR_", env_file=".env", extra="ignore"
4141
)

‎infrastructure/aws/lambda/Dockerfile‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,12 @@ RUN rm -rdf /asset/numpy/doc/ /asset/bin /asset/geos_license /asset/Misc
3030
RUN rm -rdf /asset/boto3*
3131
RUN rm -rdf /asset/botocore*
3232

33+
# Strip debug symbols from compiled C/C++ code (except for numpy.libs!)
34+
RUN cd /asset && \
35+
find . -type f -name '*.so*' \
36+
-not -path "./numpy.libs/*" \
37+
-exec strip --strip-unneeded {} \;
38+
3339
COPY infrastructure/aws/lambda/handler.py /asset/handler.py
3440

3541
CMD ["echo", "hello world"]

‎infrastructure/aws/package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
"license": "MIT",
66
"private": true,
77
"dependencies": {
8-
"cdk": "2.94.0"
8+
"cdk": "^2.177.0"
99
},
1010
"scripts": {
1111
"cdk": "cdk"

‎pyproject.toml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,12 @@ dev = [
7575
"pytest-benchmark>=5.1.0",
7676
"seaborn>=0.13.2",
7777
]
78+
deployment = [
79+
"aws-cdk-lib~=2.177.0",
80+
"constructs>=10.4.2",
81+
"pydantic-settings~=2.0",
82+
"python-dotenv>=1.0.1"
83+
]
7884

7985
[project.urls]
8086
Homepage = 'https://developmentseed.org/titiler-cmr/'

‎titiler/cmr/backend.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@
3737

3838
@cached( # type: ignore
3939
TTLCache(maxsize=100, ttl=60),
40-
key=lambda auth, daac: hashkey(auth.tokens[0]["access_token"], daac),
40+
key=lambda auth, daac: hashkey(auth.token["access_token"], daac),
4141
)
4242
def aws_s3_credential(auth: Auth, provider: str) -> Dict:
4343
"""Get AWS S3 credential through earthaccess."""

0 commit comments

Comments
 (0)