Skip to content

Commit b41b95b

Browse files
brightsparcbrightsparc
authored
Add github action workflow for build and deploy (#2)
* Adding github actions build pipeline * Updating build pipeline for staging and prod * Update to pass InputSource to pipeline. Added filter for prod * Moving drift-service-catalog to cloudformation folder
1 parent 5f67c3e commit b41b95b

File tree

13 files changed

+415
-60
lines changed

13 files changed

+415
-60
lines changed

‎.github/workflows/build-deploy-pipeline.yml

Lines changed: 255 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,255 @@
1+
# Create a unified Build and Deploy pipeline that uses approval gates
2+
# Build Model –> Deploy to staging with approval –> Deploy to prod with approval.
3+
# https://timheuer.com/blog/add-approval-workflow-to-github-actions/
4+
5+
name: Build and Deploy
6+
7+
on: [push, pull_request] # Optionally filter on branch
8+
9+
jobs:
10+
build:
11+
name: Build Model
12+
runs-on: ubuntu-latest
13+
environment:
14+
name: development
15+
defaults:
16+
run:
17+
shell: bash
18+
working-directory: ./build_pipeline
19+
steps:
20+
- name: Checkout
21+
uses: actions/checkout@v2
22+
23+
- name: Config Environment
24+
id: env-name
25+
env:
26+
PROJECT_NAME: ${{ github.event.repository.name }}
27+
run: |
28+
echo "Project name: $PROJECT_NAME"
29+
echo "::set-output name=project_name::$PROJECT_NAME"
30+
31+
- name: Setup Python
32+
uses: actions/setup-python@v2
33+
with:
34+
python-version: "3.8"
35+
architecture: "x64"
36+
37+
- name: Setup Node
38+
uses: actions/setup-node@v2
39+
with:
40+
node-version: "12"
41+
architecture: "x64"
42+
cache: npm
43+
44+
- name: Install Requirements
45+
run: |
46+
npm install -g aws-cdk # Install cdk
47+
pip install --requirement requirements.txt
48+
49+
- name: Configure AWS Credentials
50+
id: creds
51+
uses: aws-actions/configure-aws-credentials@v1
52+
with:
53+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
54+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
55+
aws-region: ${{ secrets.AWS_REGION }}
56+
role-to-assume: ${{ secrets.AWS_SAGEMAKER_ROLE }}
57+
role-duration-seconds: 1200
58+
59+
- name: Build Pipeline
60+
id: build-pipeline
61+
env:
62+
SAGEMAKER_PROJECT_NAME: ${{ steps.env-name.outputs.project_name }}
63+
SAGEMAKER_PIPELINE_NAME: ${{ steps.env-name.outputs.project_name }}-pipeline
64+
SAGEMAKER_PIPELINE_DESCRIPTION: "SageMaker pipeline created from GitHub actions"
65+
SAGEMAKER_PIPELINE_ROLE_ARN: ${{ secrets.AWS_SAGEMAKER_ROLE }}
66+
AWS_REGION: ${{ secrets.AWS_REGION }}
67+
run: |
68+
export SAGEMAKER_PROJECT_ID=`aws sagemaker describe-project --project-name $SAGEMAKER_PROJECT_NAME --query ProjectId --output text`
69+
echo "Project id: $SAGEMAKER_PROJECT_ID"
70+
export ARTIFACT_BUCKET=sagemaker-project-$SAGEMAKER_PROJECT_ID-$AWS_REGION
71+
echo "Artifact Bucket: $ARTIFACT_BUCKET"
72+
npx cdk synth --path-metadata false --asset-metadata=false > drift-pipeline.yml
73+
echo "::set-output name=pipeline_name::$SAGEMAKER_PIPELINE_NAME"
74+
75+
- name: Print template
76+
run: cat drift-pipeline.yml
77+
78+
- name: Create CFN Pipeline
79+
id: deploy-pipeline
80+
uses: aws-actions/aws-cloudformation-github-deploy@v1
81+
with:
82+
name: sagemaker-${{ steps.build-pipeline.outputs.pipeline_name }}
83+
template: ./build_pipeline/drift-pipeline.yml # Need to specify working-directory
84+
no-fail-on-empty-changeset: "1"
85+
86+
- name: Start Pipeline
87+
id: start-pipeline # TODO: Run python code that waits for pipeline to complete
88+
run: aws sagemaker start-pipeline-execution --pipeline-name ${{ steps.build-pipeline.outputs.pipeline_name }} --pipeline-parameters Name=InputSource,Value=GitHubAction#${{ github.run_number }}
89+
90+
- name: Upload template
91+
uses: actions/upload-artifact@v2
92+
with:
93+
name: drift-pipeline
94+
path: ./build_pipeline/drift-pipeline.yml
95+
96+
deploy_staging:
97+
needs: build
98+
name: Deploy to staging
99+
runs-on: ubuntu-latest
100+
environment:
101+
name: staging # Use different environment that optionally requires approval
102+
defaults:
103+
run:
104+
shell: bash
105+
working-directory: ./deployment_pipeline
106+
steps:
107+
- name: Checkout
108+
uses: actions/checkout@v2
109+
110+
- name: Config Environment
111+
id: env-name
112+
env:
113+
PROJECT_NAME: ${{ github.event.repository.name }}
114+
run: |
115+
echo "Project name: $PROJECT_NAME"
116+
echo "::set-output name=project_name::$PROJECT_NAME"
117+
118+
- name: Setup Python
119+
uses: actions/setup-python@v2
120+
with:
121+
python-version: "3.8"
122+
architecture: "x64"
123+
124+
- name: Setup Node
125+
uses: actions/setup-node@v2
126+
with:
127+
node-version: "12"
128+
architecture: "x64"
129+
cache: npm
130+
131+
- name: Install Requirements
132+
run: |
133+
npm install -g aws-cdk # Install cdk
134+
pip install --requirement requirements.txt
135+
136+
- name: Configure AWS Credentials
137+
id: creds
138+
uses: aws-actions/configure-aws-credentials@v1
139+
with:
140+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
141+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
142+
aws-region: ${{ secrets.AWS_REGION }}
143+
role-to-assume: ${{ secrets.AWS_SAGEMAKER_ROLE }}
144+
role-duration-seconds: 1200
145+
146+
- name: Build Templates
147+
id: build-templates
148+
env:
149+
SAGEMAKER_PROJECT_NAME: ${{ steps.env-name.outputs.project_name }}
150+
SAGEMAKER_EXECUTION_ROLE_ARN: ${{ secrets.AWS_SAGEMAKER_ROLE }}
151+
AWS_REGION: ${{ secrets.AWS_REGION }}
152+
run: |
153+
export SAGEMAKER_PROJECT_ID=`aws sagemaker describe-project --project-name $SAGEMAKER_PROJECT_NAME --query ProjectId --output text`
154+
echo "Project id: $SAGEMAKER_PROJECT_ID"
155+
export ARTIFACT_BUCKET=sagemaker-project-$SAGEMAKER_PROJECT_ID-$AWS_REGION
156+
echo "Artifact Bucket: $ARTIFACT_BUCKET"
157+
npx cdk synth drift-deploy-staging --path-metadata false --asset-metadata=false > drift-deploy-staging.yml
158+
159+
- name: Print template
160+
run: cat drift-deploy-staging.yml
161+
162+
- name: Deploy Staging
163+
id: deploy-pipeline
164+
uses: aws-actions/aws-cloudformation-github-deploy@v1
165+
with:
166+
name: sagemaker-${{ steps.env-name.outputs.project_name }}-deploy-staging
167+
template: ./deployment_pipeline/drift-deploy-staging.yml # Need to specify working-directory
168+
no-fail-on-empty-changeset: "1"
169+
170+
- name: Upload template
171+
uses: actions/upload-artifact@v2
172+
with:
173+
name: drift-deploy-staging
174+
path: ./deployment_pipeline/drift-deploy-staging.yml
175+
176+
deploy_prod:
177+
needs: deploy_staging
178+
name: Deploy to prod
179+
if: ${{ github.ref == 'refs/heads/main' }} # Filter to only run on main branch
180+
runs-on: ubuntu-latest
181+
environment:
182+
name: prod # Use different environment that requires approval
183+
defaults:
184+
run:
185+
shell: bash
186+
working-directory: ./deployment_pipeline
187+
steps:
188+
- name: Checkout
189+
uses: actions/checkout@v2
190+
191+
- name: Config Environment
192+
id: env-name
193+
env:
194+
PROJECT_NAME: ${{ github.event.repository.name }}
195+
run: |
196+
echo "Project name: $PROJECT_NAME"
197+
echo "::set-output name=project_name::$PROJECT_NAME"
198+
199+
- name: Setup Python
200+
uses: actions/setup-python@v2
201+
with:
202+
python-version: "3.8"
203+
architecture: "x64"
204+
205+
- name: Setup Node
206+
uses: actions/setup-node@v2
207+
with:
208+
node-version: "12"
209+
architecture: "x64"
210+
cache: npm
211+
212+
- name: Install Requirements
213+
run: |
214+
npm install -g aws-cdk # Install cdk
215+
pip install --requirement requirements.txt
216+
217+
- name: Configure AWS Credentials
218+
id: creds
219+
uses: aws-actions/configure-aws-credentials@v1
220+
with:
221+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
222+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
223+
aws-region: ${{ secrets.AWS_REGION }}
224+
role-to-assume: ${{ secrets.AWS_SAGEMAKER_ROLE }}
225+
role-duration-seconds: 1200
226+
227+
- name: Build Templates
228+
id: build-templates
229+
env:
230+
SAGEMAKER_PROJECT_NAME: ${{ steps.env-name.outputs.project_name }}
231+
SAGEMAKER_EXECUTION_ROLE_ARN: ${{ secrets.AWS_SAGEMAKER_ROLE }}
232+
AWS_REGION: ${{ secrets.AWS_REGION }}
233+
run: |
234+
export SAGEMAKER_PROJECT_ID=`aws sagemaker describe-project --project-name $SAGEMAKER_PROJECT_NAME --query ProjectId --output text`
235+
echo "Project id: $SAGEMAKER_PROJECT_ID"
236+
export ARTIFACT_BUCKET=sagemaker-project-$SAGEMAKER_PROJECT_ID-$AWS_REGION
237+
echo "Artifact Bucket: $ARTIFACT_BUCKET"
238+
npx cdk synth drift-deploy-prod --path-metadata false --asset-metadata=false > drift-deploy-prod.yml
239+
240+
- name: Print Template
241+
run: cat drift-deploy-prod.yml
242+
243+
- name: Deploy Prod
244+
id: deploy-pipeline
245+
uses: aws-actions/aws-cloudformation-github-deploy@v1
246+
with:
247+
name: sagemaker-${{ steps.env-name.outputs.project_name }}-deploy-prod
248+
template: ./deployment_pipeline/drift-deploy-prod.yml # Need to specify working-directory
249+
no-fail-on-empty-changeset: "1"
250+
251+
- name: Upload template
252+
uses: actions/upload-artifact@v2
253+
with:
254+
name: drift-deploy-prod
255+
path: ./deployment_pipeline/drift-deploy-prod.yml

‎.github/workflows/publish-template.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,12 @@ name: Publish Template to S3
33
on:
44
push:
55
branches: [ main ]
6-
pull_request:
7-
branches: [ main ]
86

97
jobs:
108
synth-and-publish:
119
runs-on: ubuntu-latest
10+
environment:
11+
name: aws # Target only the AWS environment to publish CFN templates
1212
defaults:
1313
run:
1414
shell: bash
@@ -19,8 +19,8 @@ jobs:
1919
- name: Setup Python environment
2020
uses: actions/setup-python@v2
2121
with:
22-
python-version: "3.8" # Version range or exact version of a Python version to use, using SemVer's version range syntax
23-
architecture: "x64" # optional x64 or x86. Defaults to x64 if not specified
22+
python-version: "3.8"
23+
architecture: "x64"
2424

2525
- name: Setup Node
2626
uses: actions/setup-node@v2
@@ -47,7 +47,7 @@ jobs:
4747
env:
4848
BUCKET_NAME: ${{ secrets.BUCKET_NAME }}
4949
BUCKET_PREFIX: ${{ secrets.BUCKET_PREFIX }}
50-
run: cdk synth drift-service-catalog --path-metadata false -c drift:ArtifactBucket=$BUCKET_NAME -c drift:ArtifactBucketPrefix=$BUCKET_PREFIX > drift-service-catalog.yml
50+
run: cdk synth drift-service-catalog --path-metadata false -c drift:ArtifactBucket=$BUCKET_NAME -c drift:ArtifactBucketPrefix=$BUCKET_PREFIX > cloudformation/drift-service-catalog.yml
5151

5252
- name: Publish Assets to S3
5353
env:
@@ -62,6 +62,6 @@ jobs:
6262
path: cdk.out/*.template.json
6363

6464
- name: Print Template
65-
run: cat drift-service-catalog.yml
65+
run: cat cloudformation/drift-service-catalog.yml
6666

6767

‎ACTIONS.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# AWS SageMaker Workflow for GitHub Actions
2+
3+
This template repository contains a sample application and sample GitHub Actions workflow files for continuously deploying both application code and infrastructure as code with GitHub Actions.
4+
5+
This MLOps workflow demonstrates training and evaluating a machine learning model to predict taxi fare from the public [New York City Taxi dataset](https://registry.opendata.aws/nyc-tlc-trip-records-pds/) deployed with Amazon SageMaker.
6+
7+
This repository contains a number of start workflow files for GitHub Actions:
8+
1. [build-pipeline.yml](./build-pipeline.yml) runs when a pull request is opened or updated. This workflow creates or updates the Sagemaker Pipeline, and starts an execution.
9+
1. [deploy-staging.yml](./deploy-staging.yml) runs when a new commit is pushed to the master branch for the `staging` environment. This workflow deploys the SageMaker endpoint for staging. (WIP)
10+
1. [deploy-prod.yml](./deploy-prod.yml) runs when a new commit is pushed to the master branch for the `production` environment. This workflow deploys the Sagemaker endpoint for production. (WIP)
11+
1. [publish-template.yml](./publish-template.yml) runs when a new commit is pushed to the master branch for the `public` environment. Synths and uploads the CFN templates and build & deploy assets for launching the stack.
12+
13+
## Create a GitHub repository from this template
14+
15+
Click the "Use this template" button above to create a new repository from this template.
16+
17+
Clone your new repository, and deploy the IAM resources needed to enable GitHub Actions to deploy CloudFormation templates:
18+
19+
```
20+
aws cloudformation deploy \
21+
--stack-name amazon-sagemaker-workflow-for-github-actions \
22+
--template-file cloudformation/github-actions-setup.yml \
23+
--capabilities CAPABILITY_NAMED_IAM \
24+
--region us-east-1
25+
```
26+
You can review the permissions that your repository's GitHub Actions deployment workflow will have in the [setup.yml](cloudformation-templates/setup.yml) CloudFormation template.
27+
28+
Retrieve the IAM access key credentials that GitHub Actions will use for deployments:
29+
```
30+
aws secretsmanager get-secret-value \
31+
--secret-id github-actions-sagemaker \
32+
--region us-east-1 \
33+
--query SecretString \
34+
--output text
35+
```
36+
37+
The GitHub Actions workflow has three stages: Model build, Deploy to staging, and Deploy to production. Each has their on [environment](https://docs.github.com/en/actions/reference/environments) which secrets and optional protection rules.
38+
1. `development` This is the environment in which runs your Model Build and starts the SageMaker pipeline execution, and on completion it will publish a model to the Registry. It is recommend you run this on `pull_request` and `push`.
39+
1. `staging` This second stage deploys your Staging endpoint. It is recommend you run this on commit to the `development` branch and configure a *protection rule* to continue after you have approved the model in the SageMaker Model Registry.
40+
1. `prod` This final stage deploys you Production Endpoint. It is recommend you this on commit the `main` branch with a *protection rule* to require approval after Staging endpoint has been tested.
41+
42+
For each of the environments you will require setting up the following secrets.
43+
1. Create a secret named `AWS_REGION` defaulted to region `us-east-1`
44+
1. Create a secret named `AWS_ACCESS_KEY_ID` containing the `AccessKeyId` value returned above.
45+
1. Create a secret named `AWS_SECRET_ACCESS_KEY` containing in the `SecretAccessKey` value returned above.
46+
1. Create a secret named `AWS_SAGEMAKER_ROLE` containing the `SageMakerRoleArn` output in the setup stack.
47+
48+
If you configure *protection rules* for you environments, you will need to click **Review deployments** to approve the next stage as show below:
49+
50+
![Execution Role](docs/github-actions-workflow.png)
51+
52+
When the workflow successfully completes, you will have both Endpoints deployed in staging and production, with drift detection enable which will trigger re-training on drift.

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Once complete, you can Train and Deploy machine learning models, and send traffi
1515

1616
Use this following AWS CloudFormation quick start to create a custom [SageMaker MLOps project](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-projects-templates-custom.html) template in the [AWS Service Catalog](https://aws.amazon.com/servicecatalog/) and configure the portfolio and products so you can launch the project from within your Studio domain.
1717

18-
[![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?templateUrl=https%3A%2F%2Famazon-sagemaker-safe-deployment-pipeline.s3.amazonaws.com%2Fdrift-pipeline%2Fdrift-service-catalog.yml&stackName=drift-pipeline&param_ExecutionRoleArn=&param_PortfolioName=SageMaker%20Organization%20Templates&param_PortfolioOwner=administrator&param_ProductVersion=1.0)
18+
[![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?templateUrl=https%3A%2F%2Famazon-sagemaker-safe-deployment-pipeline.s3.amazonaws.com%2Fdrift-pipeline%2Fcloudformation%2Fdrift-service-catalog.yml&stackName=drift-pipeline&param_ExecutionRoleArn=&param_PortfolioName=SageMaker%20Organization%20Templates&param_PortfolioOwner=administrator&param_ProductVersion=1.0)
1919

2020
Follow are the list of the parameters.
2121

0 commit comments

Comments
 (0)