Skip to content

docs(build_recipes): add cicd page #7176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 15, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/build_recipes/cicd-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: CI/CD Integration
description: Automate Lambda function builds and deployments
---

<!-- markdownlint-disable MD043 -->

Automate your Lambda function builds and deployments using popular CI/CD platforms. These examples show how to build and deploy Lambda functions with Powertools for AWS with proper cross-platform compatibility and deploy them reliably.

## GitHub Actions

**GitHub Actions** provides a powerful, integrated CI/CD platform that runs directly in your GitHub repository. It offers excellent integration with AWS services, supports matrix builds for testing multiple configurations, and provides a rich ecosystem of pre-built actions.

=== "Modern AWS Lambda deploy action"

```yaml
--8<-- "examples/build_recipes/cicd/github-actions/deploy-modern.yml"
```

=== "Multi-environment deployment"

```yaml
--8<-- "examples/build_recipes/cicd/github-actions/deploy-multi-env.yml"
```

=== "Simple source code deployment"

```yaml
--8<-- "examples/build_recipes/cicd/github-actions/deploy-simple.yml"
```

=== "S3 deployment method"

```yaml
--8<-- "examples/build_recipes/cicd/github-actions/deploy-s3.yml"
```

=== "Build tool integration"

```yaml
--8<-- "examples/build_recipes/cicd/github-actions/deploy-build-tools.yml"
```

## AWS CodeBuild

**AWS CodeBuild** is a fully managed build service that compiles source code, runs tests, and produces deployment packages. It integrates seamlessly with other AWS services and provides consistent build environments with automatic scaling.

=== "Basic CodeBuild Configuration"

```yaml
--8<-- "examples/build_recipes/cicd/codebuild/buildspec.yml"
```

## Best Practices for CI/CD

1. **Use Linux runners** (ubuntu-latest) to ensure Lambda compatibility
2. **Cache dependencies** to speed up builds (uv, poetry cache, pip cache)
3. **Run tests first** before building deployment packages
4. **Use matrix builds** to test multiple Python versions or configurations
5. **Implement proper secrets management** with GitHub Secrets or AWS Parameter Store
6. **Add deployment gates** for production environments
7. **Monitor deployment success** with CloudWatch metrics and alarms

???+ tip "Performance Optimization"
- Use **uv** for fastest dependency installation in CI/CD
- **Cache virtual environments** between builds when possible
- **Parallelize builds** for multiple environments
- **Use container images** for complex dependencies or large packages
131 changes: 131 additions & 0 deletions examples/build_recipes/cicd/codebuild/buildspec-advanced.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
version: 0.2

env:
variables:
PYTHON_VERSION: "3.13"
BUILD_STAGE: "build"
parameter-store:
POWERTOOLS_VERSION: "/build/powertools-version"

batch:
fast-fail: false
build-list:
- identifier: test
env:
variables:
BUILD_STAGE: "test"
- identifier: build_dev
env:
variables:
BUILD_STAGE: "build"
ENVIRONMENT: "dev"
depend-on:
- test
- identifier: build_prod
env:
variables:
BUILD_STAGE: "build"
ENVIRONMENT: "prod"
depend-on:
- test

phases:
install:
runtime-versions:
python: $PYTHON_VERSION
commands:
- echo "Build stage: $BUILD_STAGE, Environment: $ENVIRONMENT"
- pip install --upgrade pip uv

pre_build:
commands:
- |
if [ "$BUILD_STAGE" = "test" ]; then
echo "Installing test dependencies..."
uv venv test-env
source test-env/bin/activate
uv pip install aws-lambda-powertools[all]==$POWERTOOLS_VERSION pytest pytest-cov
cp -r src/ test-src/
else
echo "Installing build dependencies..."
uv venv build-env
source build-env/bin/activate
uv pip install aws-lambda-powertools[all]==$POWERTOOLS_VERSION
uv pip install pydantic requests
fi

build:
commands:
- |
if [ "$BUILD_STAGE" = "test" ]; then
echo "Running tests..."
source test-env/bin/activate
cd test-src
pytest tests/ --cov=. --cov-report=xml --cov-report=term
echo "Tests completed successfully"
else
echo "Building deployment package for $ENVIRONMENT..."
source build-env/bin/activate

# Create environment-specific package
mkdir -p package-$ENVIRONMENT/
cp -r build-env/lib/python*/site-packages/* package-$ENVIRONMENT/
cp -r src/* package-$ENVIRONMENT/

# Environment-specific optimizations
if [ "$ENVIRONMENT" = "prod" ]; then
echo "Applying production optimizations..."
find package-$ENVIRONMENT/ -name "*.pyc" -delete
find package-$ENVIRONMENT/ -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
find package-$ENVIRONMENT/ -name "tests" -type d -exec rm -rf {} + 2>/dev/null || true
find package-$ENVIRONMENT/ -name "*.dist-info" -type d -exec rm -rf {} + 2>/dev/null || true
fi

# Create deployment ZIP
cd package-$ENVIRONMENT && zip -r ../lambda-$ENVIRONMENT.zip . && cd ..

echo "Package size for $ENVIRONMENT: $(du -sh lambda-$ENVIRONMENT.zip)"
fi

post_build:
commands:
- |
if [ "$BUILD_STAGE" = "build" ]; then
echo "Deploying to $ENVIRONMENT environment..."

# Deploy to environment-specific function
aws lambda update-function-code \
--function-name powertools-app-$ENVIRONMENT \
--zip-file fileb://lambda-$ENVIRONMENT.zip \
--region $AWS_DEFAULT_REGION

# Update environment-specific configuration
LOG_LEVEL="INFO"
if [ "$ENVIRONMENT" = "dev" ]; then
LOG_LEVEL="DEBUG"
fi

aws lambda update-function-configuration \
--function-name powertools-app-$ENVIRONMENT \
--environment Variables="{
ENVIRONMENT=$ENVIRONMENT,
POWERTOOLS_SERVICE_NAME=powertools-app-$ENVIRONMENT,
POWERTOOLS_METRICS_NAMESPACE=MyApp/$ENVIRONMENT,
POWERTOOLS_LOG_LEVEL=$LOG_LEVEL
}" \
--region $AWS_DEFAULT_REGION

echo "Deployment to $ENVIRONMENT completed successfully!"
fi

artifacts:
files:
- lambda-*.zip
- coverage.xml
name: lambda-artifacts-$(date +%Y-%m-%d-%H-%M-%S)

cache:
paths:
- 'build-env/**/*'
- 'test-env/**/*'

85 changes: 85 additions & 0 deletions examples/build_recipes/cicd/codebuild/buildspec.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
version: 0.2

env:
variables:
PYTHON_VERSION: "3.13"
POWERTOOLS_VERSION: "3.18.0"
parameter-store:
FUNCTION_NAME: "/lambda/powertools-app/function-name"

phases:
install:
runtime-versions:
python: $PYTHON_VERSION
commands:
- echo "Installing build dependencies..."
- pip install --upgrade pip
- pip install uv poetry # Install fast package managers

pre_build:
commands:
- echo "Pre-build phase started on $(date)"
- echo "Python version: $(python --version)"
- echo "Installing application dependencies..."

# Use uv for fast dependency installation
- uv venv build-env
- source build-env/bin/activate
- uv pip install aws-lambda-powertools[all]==$POWERTOOLS_VERSION
- uv pip install pydantic requests

build:
commands:
- echo "Build started on $(date)"
- echo "Creating deployment package..."

# Create optimized deployment package
- mkdir -p package/
- cp -r build-env/lib/python*/site-packages/* package/
- cp -r src/* package/

# Remove unnecessary files to reduce package size
- find package/ -name "*.pyc" -delete
- find package/ -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
- find package/ -name "tests" -type d -exec rm -rf {} + 2>/dev/null || true
- find package/ -name "*.dist-info" -type d -exec rm -rf {} + 2>/dev/null || true

# Create deployment ZIP
- cd package && zip -r ../lambda-deployment.zip . && cd ..

# Show package info
- echo "Package size: $(du -sh lambda-deployment.zip)"
- echo "Package contents:"
- unzip -l lambda-deployment.zip | head -20

post_build:
commands:
- echo "Build completed on $(date)"
- echo "Deploying Lambda function..."

# Deploy to Lambda
- aws lambda update-function-code \
--function-name $FUNCTION_NAME \
--zip-file fileb://lambda-deployment.zip \
--region $AWS_DEFAULT_REGION

# Update environment variables
- aws lambda update-function-configuration \
--function-name $FUNCTION_NAME \
--environment Variables="{
POWERTOOLS_SERVICE_NAME=powertools-codebuild,
POWERTOOLS_METRICS_NAMESPACE=MyApp/CodeBuild,
POWERTOOLS_LOG_LEVEL=INFO
}" \
--region $AWS_DEFAULT_REGION

- echo "Deployment completed successfully!"

artifacts:
files:
- lambda-deployment.zip
name: lambda-deployment-$(date +%Y-%m-%d-%H-%M-%S)

cache:
paths:
- 'build-env/**/*' # Cache virtual environment for faster builds
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
name: Deploy with Different Build Tools

on:
push:
branches: [main]

jobs:
deploy-poetry:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0

- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: '3.13'

- name: Install Poetry
run: pip install --upgrade pip poetry

- name: Build with Poetry
run: |
# Create deployment directory
mkdir -p poetry-deploy/

# Export and install dependencies
poetry export -f requirements.txt --output requirements.txt --without-hashes
pip install --platform manylinux2014_x86_64 --only-binary=:all: \
--python-version 3.13 -r requirements.txt -t poetry-deploy/

# Copy source code
cp -r src/* poetry-deploy/

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@209f2a4450bb4b277e1dedaff40ad2fd8d4d0a4c # v4.3.0
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: us-east-1

- name: Deploy Poetry build
uses: aws-actions/aws-lambda-deploy@246115fccc1ad110c97f729696574d09eb5c690d
with:
function-name: powertools-poetry-function
code-artifacts-dir: poetry-deploy/
handler: app.lambda_handler
runtime: python3.13
environment: '{"POWERTOOLS_SERVICE_NAME":"powertools-poetry","POWERTOOLS_METRICS_NAMESPACE":"MyApp","POWERTOOLS_LOG_LEVEL":"INFO"}'
role: ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }}

deploy-uv:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0

- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: '3.13'

- name: Build with uv (fastest)
run: |
# Install uv
pip install uv

# Create deployment directory
mkdir -p uv-deploy/

# Install dependencies with uv (much faster)
uv pip install \
--target uv-deploy/ \
--python-version 3.13 \
--platform manylinux2014_x86_64 --only-binary=:all: \
aws-lambda-powertools[all] pydantic requests

# Copy source code
cp -r src/* uv-deploy/

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@209f2a4450bb4b277e1dedaff40ad2fd8d4d0a4c # v4.3.0
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: us-east-1

- name: Deploy uv build
uses: aws-actions/aws-lambda-deploy@246115fccc1ad110c97f729696574d09eb5c690d # v1.0.1
with:
function-name: powertools-uv-function
code-artifacts-dir: uv-deploy/
handler: app.lambda_handler
runtime: python3.13
environment: '{"POWERTOOLS_SERVICE_NAME":"powertools-uv","POWERTOOLS_METRICS_NAMESPACE":"MyApp","POWERTOOLS_LOG_LEVEL":"INFO"}'
role: ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }}
Loading