diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index de0c36b21e0..f0ea1cbf495 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -34,7 +34,7 @@ repos: entry: poetry run cfn-lint language: system types: [yaml] - exclude: examples/homepage/install/.*?/serverless\.yml$ + exclude: examples/build_recipes/* files: examples/.* - repo: https://github.com/rhysd/actionlint rev: "fd7ba3c382e13dcc0248e425b4cbc3f1185fa3ee" # v1.6.24 diff --git a/docs/build_recipes/build-with-cdk.md b/docs/build_recipes/build-with-cdk.md new file mode 100644 index 00000000000..502d1671daa --- /dev/null +++ b/docs/build_recipes/build-with-cdk.md @@ -0,0 +1,133 @@ +--- +title: Build with CDK +description: Package Lambda functions using AWS CDK for infrastructure as code +--- + + + +The **AWS CDK (Cloud Development Kit)** allows you to define cloud infrastructure using familiar programming languages like Python, TypeScript, or Java. It provides type safety, IDE support, and the ability to create reusable constructs, making it perfect for complex infrastructure requirements and teams that prefer code over YAML. + +Learn more at [AWS CDK documentation](https://docs.aws.amazon.com/cdk/){target="_blank"}. + +## Basic CDK setup with Python + +CDK uses the concept of **Apps**, **Stacks**, and **Constructs** to organize infrastructure. A CDK app contains one or more stacks, and each stack contains constructs that represent AWS resources. + +### Project structure + +```bash +my-lambda-cdk/ +├── app.py # CDK app entry point +├── cdk.json # CDK configuration +├── requirements.txt # CDK dependencies +├── src/ +│ └── lambda_function.py # Lambda function code +└── stacks/ + └── lambda_stack.py # Stack definition (optional) +``` + +### Key CDK concepts for Lambda + +| Concept | Description | Lambda Usage | +|---------|-------------|--------------| +| **App** | Root construct, contains stacks | Entry point for your Lambda infrastructure | +| **Stack** | Unit of deployment | Groups related Lambda functions and resources | +| **Construct** | Reusable cloud component | Lambda function, API Gateway, DynamoDB table | +| **Asset** | Local files bundled with deployment | Lambda function code, layers | + +### Prerequisites + +Before starting, ensure you have: + +```bash +--8<-- "examples/build_recipes/cdk/basic/setup-cdk.sh" +``` + +### Basic implementation + +=== "app.py" + + ```python + --8<-- "examples/build_recipes/cdk/basic/app.py" + ``` + +=== "cdk.json" + + ```json + --8<-- "examples/build_recipes/cdk/basic/cdk.json" + ``` + +=== "requirements.txt" + + ```txt + --8<-- "examples/build_recipes/cdk/basic/requirements.txt" + ``` + +=== "src/lambda_function.py" + + ```python + --8<-- "examples/build_recipes/cdk/basic/src/lambda_function.py" + ``` + +=== "build-cdk.sh" + + ```bash + --8<-- "examples/build_recipes/cdk/basic/build-cdk.sh" + ``` + +### CDK bundling options + +CDK provides several ways to handle Lambda function dependencies: + +| Method | Description | Best For | +|--------|-------------|----------| +| **Inline bundling** | CDK bundles dependencies automatically | Simple functions with few dependencies | +| **Docker bundling** | Uses Docker for consistent builds | Complex dependencies, cross-platform builds | +| **Pre-built assets** | Upload pre-packaged ZIP files | Custom build processes, CI/CD integration | +| **Lambda Layers** | Separate dependencies from code | Shared dependencies across functions | + +### Common CDK commands + +```bash +--8<-- "examples/build_recipes/cdk/basic/cdk-commands.sh" +``` + +## Advanced CDK with multiple stacks + +Multi-environment CDK setup with separate stacks, DynamoDB integration, and SQS message processing using BatchProcessor. + +=== "stacks/powertools_cdk_stack.py" + + ```python + --8<-- "examples/build_recipes/cdk/multi-stack/stacks/powertools_cdk_stack.py" + ``` + +=== "cdk.json" + + ```json + --8<-- "examples/build_recipes/cdk/multi-stack/cdk.json" + ``` + +=== "app_multi_stack.py" + + ```python + --8<-- "examples/build_recipes/cdk/multi-stack/app_multi_stack.py" + ``` + +=== "src/app/api.py" + + ```python + --8<-- "examples/build_recipes/cdk/multi-stack/src/app/api.py" + ``` + +=== "src/worker/worker.py" + + ```python + --8<-- "examples/build_recipes/cdk/multi-stack/src/worker/worker.py" + ``` + +=== "deploy-environments.sh" + + ```bash + --8<-- "examples/build_recipes/cdk/multi-stack/deploy-environments.sh" + ``` diff --git a/docs/build_recipes/build-with-pants.md b/docs/build_recipes/build-with-pants.md new file mode 100644 index 00000000000..171bbb256db --- /dev/null +++ b/docs/build_recipes/build-with-pants.md @@ -0,0 +1,62 @@ +--- +title: Build with Pants +description: Package Lambda functions using Pants for monorepos and complex projects +--- + + + +**Pants** is a powerful build system designed for large codebases and monorepos. It provides incremental builds, dependency inference, and advanced caching mechanisms. Ideal for organizations with complex Python projects that need fine-grained build control and optimization. + +## Setup + +=== "pants.toml" + + ```toml + --8<-- "examples/build_recipes/pants/basic_pants/pants.toml" + ``` + +=== "BUILD" + + ```python + --8<-- "examples/build_recipes/pants/basic_pants/BUILD" + ``` + +=== "app.py" + + ```python + --8<-- "examples/build_recipes/pants/basic_pants/app_pants.py" + ``` + +=== "build-pants.sh" + + ```bash + --8<-- "examples/build_recipes/pants/basic_pants/build-pants.sh" + ``` + +## Advanced Pants with multiple targets + +Pants excels at managing complex projects with multiple Lambda functions that share dependencies. This approach provides significant benefits for monorepo architectures and microservices. + +=== "BUILD" + + ```python + --8<-- "examples/build_recipes/pants/multi-target/BUILD" + ``` + +=== "app/handler.py" + + ```python + --8<-- "examples/build_recipes/pants/multi-target/app/handler.py" + ``` + +=== "worker/worker_pants.py" + + ```python + --8<-- "examples/build_recipes/pants/multi-target/worker/worker_pants.py" + ``` + +=== "build-pants-multi.sh" + + ```bash + --8<-- "examples/build_recipes/pants/multi-target/build-pants-multi.sh" + ``` diff --git a/docs/build_recipes/build-with-pip.md b/docs/build_recipes/build-with-pip.md new file mode 100644 index 00000000000..160fa9093dd --- /dev/null +++ b/docs/build_recipes/build-with-pip.md @@ -0,0 +1,80 @@ +--- +title: Build with pip +description: Package Lambda functions using pip - simple and universal +--- + + + +**pip** is Python's standard package installer - simple, reliable, and available everywhere. Perfect for straightforward Lambda functions where you need basic dependency management without complex workflows. + +???+ warning "Cross-platform compatibility" + Always use `--platform manylinux2014_x86_64` and `--only-binary=:all:` flags when building on non-Linux systems to ensure Lambda compatibility. This forces pip to download Linux-compatible wheels instead of compiling from source. + +## Basic setup + +=== "requirements.txt" + + ```bash + --8<-- "examples/build_recipes/pip/requirements.txt" + ``` + +=== "app_pip.py" + + ```python + --8<-- "examples/build_recipes/pip/app_pip.py" + ``` + +=== "build.sh" + + ```bash + --8<-- "examples/build_recipes/pip/build.sh" + ``` + +## Advanced pip with Lambda Layers + +Optimize your deployment by using Lambda Layers for Powertools for AWS: + +=== "requirements-layer.txt" + + ```bash + --8<-- "examples/build_recipes/pip/requirements-layer.txt" + ``` + +=== "requirements-app.txt" + + ```bash + --8<-- "examples/build_recipes/pip/requirements-app.txt" + ``` + +=== "app_pip.py" + + ```python + --8<-- "examples/build_recipes/pip/app_pip.py" + ``` + +=== "build-with-layer.sh" + + ```bash + --8<-- "examples/build_recipes/pip/build-with-layer.sh" + ``` + +## Cross-platform builds + +Build packages for different Lambda architectures using platform-specific wheels: + +=== "Multi-architecture build" + + ```bash + --8<-- "examples/build_recipes/pip/build-cross-platform.sh" + ``` + +### Platform compatibility + +| Platform Flag | Lambda Architecture | Use Case | +|---------------|-------------------|----------| +| `manylinux2014_x86_64` | x86_64 | Standard Lambda functions | +| `manylinux2014_aarch64` | arm64 | Graviton2-based functions (lower cost) | + +???+ tip "Architecture selection" + - **x86_64**: Broader package compatibility, more mature ecosystem + - **arm64**: Up to 20% better price-performance, newer architecture diff --git a/docs/build_recipes/build-with-poetry.md b/docs/build_recipes/build-with-poetry.md new file mode 100644 index 00000000000..43690c2cfef --- /dev/null +++ b/docs/build_recipes/build-with-poetry.md @@ -0,0 +1,87 @@ +--- +title: Build with Poetry +description: Package Lambda functions using Poetry for modern dependency management +--- + + + +**Poetry** is a modern Python dependency manager that handles packaging, dependency resolution, and virtual environments. It uses lock files to ensure reproducible builds and provides excellent developer experience with semantic versioning. + +???+ warning "Cross-platform compatibility" + When building on non-Linux systems, use `pip install` with `--platform manylinux2014_x86_64` and `--only-binary=:all:` flags after exporting requirements from Poetry. This ensures Lambda-compatible wheels are installed. + +## Setup Poetry + +???+ info "Prerequisites" + - **Poetry 2.0+** required for optimal performance and latest features + - Initialize a new project with `poetry new my-lambda-project` or `poetry init` in existing directory + - Project name in `pyproject.toml` can be customized to match your preferences + - See [Poetry documentation](https://python-poetry.org/docs/basic-usage/){target="_blank"} for detailed project setup guide + +=== "pyproject.toml" + + ```toml + --8<-- "examples/build_recipes/poetry/pyproject.toml" + ``` + +=== "app.py" + + ```python + --8<-- "examples/build_recipes/poetry/app_poetry.py" + ``` + +=== "build-with-poetry.sh" + + ```bash + --8<-- "examples/build_recipes/poetry/build-with-poetry.sh" + ``` + +### Alternative: Poetry-only build (not recommended for production) + +For development or when cross-platform compatibility is not a concern: + +=== "build-poetry-native.sh" + + ```bash + --8<-- "examples/build_recipes/poetry/build-poetry-native.sh" + ``` + +## Cross-platform builds with Poetry + +Build packages for different Lambda architectures by combining Poetry's dependency management with pip's platform-specific installation: + +=== "Multi-architecture build" + + ```bash + --8<-- "examples/build_recipes/poetry/build-poetry-cross-platform.sh" + ``` + +### Poetry build methods comparison + +| Method | Cross-platform Safe | Speed | Reproducibility | Recommendation | +|--------|-------------------|-------|-----------------|----------------| +| **Poetry + pip** | ✅ Yes | Fast | High | ✅ Recommended | +| **Poetry native** | ❌ No | Fastest | Medium | ⚠️ Development only | +| **Poetry + Docker** | ✅ Yes | Slower | Highest | ✅ Complex dependencies | + +???+ tip "Poetry best practices for Lambda" + - Always use `poetry export` to generate requirements.txt for deployment + - Use `--without-hashes` flag to avoid pip compatibility issues + - Combine with `pip install --platform` for cross-platform builds + - Keep `poetry.lock` in version control for reproducible builds + +## Poetry with Docker for consistent builds + +Use Docker to ensure consistent builds across different development environments and avoid platform-specific dependency issues. + +=== "Dockerfile" + + ```dockerfile title="Dockerfile.poetry" + --8<-- "examples/build_recipes/poetry/Dockerfile.poetry" + ``` + +=== "build-with-poetry-docker.sh" + + ```bash + --8<-- "examples/build_recipes/poetry/build-with-poetry-docker.sh" + ``` diff --git a/docs/build_recipes/build-with-sam.md b/docs/build_recipes/build-with-sam.md new file mode 100644 index 00000000000..0166d1b85d5 --- /dev/null +++ b/docs/build_recipes/build-with-sam.md @@ -0,0 +1,104 @@ +--- +title: Build with SAM +description: Package Lambda functions using AWS SAM for serverless applications +--- + + + +**AWS SAM (Serverless Application Model)** is AWS's framework for building serverless applications using CloudFormation templates. It provides local testing capabilities, built-in best practices, and seamless integration with AWS services, making it the go-to choice for AWS-native serverless development. + +SAM automatically resolves multi-architecture compatibility issues by building functions inside Lambda-compatible containers (`--use-container` flag), ensuring dependencies are installed with the correct architecture and glibc versions for the Lambda runtime environment. This eliminates the common problem of architecture mismatches when building on macOS/Windows. + +Learn more at [AWS SAM documentation](https://docs.aws.amazon.com/serverless-application-model/){target="_blank"}. + +## SAM without Layers (All-in-one package) + +Simple approach where all dependencies are packaged with the function code: + +=== "template.yaml" + + ```yaml + --8<-- "examples/build_recipes/sam/no-layers/template.yaml" + ``` + +=== "requirements.txt" + + ```txt + --8<-- "examples/build_recipes/sam/no-layers/requirements.txt" + ``` + +=== "src/app_sam_no_layer.py" + + ```python + --8<-- "examples/build_recipes/sam/no-layers/src/app_sam_no_layer.py" + ``` + +=== "build-sam-no-layers.sh" + + ```bash + --8<-- "examples/build_recipes/sam/no-layers/build-sam-no-layers.sh" + ``` + +## SAM with Layers (Optimized approach) + +Optimized approach using Lambda Layers to separate dependencies from application code. This example demonstrates: + +* **Public Powertools for AWS Lambda layer** - Uses AWS-managed layer ARN for better performance and maintenance +* **Custom dependencies layer** - Separates application-specific dependencies + +=== "template.yaml" + + ```yaml + --8<-- "examples/build_recipes/sam/with-layers/template.yaml" + ``` + +=== "layers/dependencies/requirements.txt" + + ```txt + --8<-- "examples/build_recipes/sam/with-layers/layers/dependencies/requirements.txt" + ``` + +=== "src/app/app_sam_layer.py" + + ```python + --8<-- "examples/build_recipes/sam/with-layers/src/app/app_sam_layer.py" + ``` + +=== "src/worker/worker_sam_layer.py" + + ```python + --8<-- "examples/build_recipes/sam/with-layers/src/worker/worker_sam_layer.py" + ``` + +=== "samconfig.toml" + + ```toml + --8<-- "examples/build_recipes/sam/with-layers/samconfig.toml" + ``` + +=== "build-sam-with-layers.sh" + + ```bash + --8<-- "examples/build_recipes/sam/with-layers/build-sam-with-layers.sh" + ``` + +## Comparison: with vs without Layers + +| Aspect | Without Layers | With Layers | +|--------|----------------|-------------| +| **Deployment Speed** | Slower (uploads all deps each time) | Faster (layers cached, only app code changes) | +| **Package Size** | Larger function packages | Smaller function packages | +| **Cold Start** | Slightly faster (everything in one place) | Slightly slower (layer loading overhead) | +| **Reusability** | No sharing between functions | Layers shared across functions | +| **Complexity** | Simple, single package | More complex, multiple components | +| **Best For** | Single function, simple apps | Multiple functions, shared dependencies | + +## Advanced SAM with multiple environments + +Configure different environments (dev, staging, prod) with environment-specific settings and layer references. This example demonstrates how to use parameters, mappings, and conditions to create flexible, multi-environment deployments. + +=== "template.yaml" + + ```yaml + --8<-- "examples/build_recipes/sam/multi-env/template.yaml" + ``` diff --git a/docs/build_recipes/build-with-uv.md b/docs/build_recipes/build-with-uv.md new file mode 100644 index 00000000000..a1250980888 --- /dev/null +++ b/docs/build_recipes/build-with-uv.md @@ -0,0 +1,65 @@ +--- +title: Build with uv +description: Package Lambda functions using uv for extremely fast builds +--- + + + +**uv** is an extremely fast Python package manager written in Rust, designed as a drop-in replacement for pip and pip-tools. It offers 10-100x faster dependency resolution and installation, making it ideal for CI/CD pipelines and performance-critical builds. Learn more at [docs.astral.sh/uv/](https://docs.astral.sh/uv/){target="_blank"}. + +???+ warning "Cross-platform compatibility" + Use `uv pip install` with `--platform manylinux2014_x86_64` and `--only-binary=:all:` flags when building on non-Linux systems. This ensures Lambda-compatible wheels are downloaded instead of compiling from source. + +## Setup uv + +=== "pyproject.toml" + + ```toml + --8<-- "examples/build_recipes/uv/pyproject.toml" + ``` + +=== "app_uv.py" + + ```python + --8<-- "examples/build_recipes/uv/app_uv.py" + ``` + +=== "build-uv.sh" + + ```bash + --8<-- "examples/build_recipes/uv/build-uv.sh" + ``` + +## uv with lock file for reproducible builds + +Generate and use lock files to ensure exact dependency versions across all environments and team members. + +=== "build-uv-locked.sh" + + ```bash + --8<-- "examples/build_recipes/uv/build-uv-locked.sh" + ``` + +## Cross-platform builds with uv + +Build packages for different Lambda architectures using uv's platform-specific installation: + +=== "Multi-architecture build" + + ```bash + --8<-- "examples/build_recipes/uv/build-uv-cross-platform.sh" + ``` + +### uv performance advantages + +| Feature | uv | pip | Benefit | +|---------|----|----|---------| +| **Dependency resolution** | Rust-based solver | Python-based | 10-100x faster | +| **Parallel downloads** | Built-in | Limited | Faster package installation | +| **Lock file generation** | `uv lock` | Requires pip-tools | Reproducible builds | +| **Virtual environments** | `uv venv` | Separate venv tool | Integrated workflow | + +???+ tip "uv best practices for Lambda" + - Use `uv lock` for reproducible builds across environments + - Leverage `uv export` to generate requirements.txt for deployment + - Use `--frozen` flag in CI/CD to ensure exact dependency versions diff --git a/docs/build_recipes/cicd-integration.md b/docs/build_recipes/cicd-integration.md new file mode 100644 index 00000000000..69e6fb261a2 --- /dev/null +++ b/docs/build_recipes/cicd-integration.md @@ -0,0 +1,68 @@ +--- +title: CI/CD Integration +description: Automate Lambda function builds and deployments +--- + + + +Automate your Lambda function builds and deployments using popular CI/CD platforms. These examples show how to build and deploy Lambda functions with Powertools for AWS with proper cross-platform compatibility and deploy them reliably. + +## GitHub Actions + +**GitHub Actions** provides a powerful, integrated CI/CD platform that runs directly in your GitHub repository. It offers excellent integration with AWS services, supports matrix builds for testing multiple configurations, and provides a rich ecosystem of pre-built actions. + +=== "Modern AWS Lambda deploy action" + + ```yaml + --8<-- "examples/build_recipes/cicd/github-actions/deploy-modern.yml" + ``` + +=== "Multi-environment deployment" + + ```yaml + --8<-- "examples/build_recipes/cicd/github-actions/deploy-multi-env.yml" + ``` + +=== "Simple source code deployment" + + ```yaml + --8<-- "examples/build_recipes/cicd/github-actions/deploy-simple.yml" + ``` + +=== "S3 deployment method" + + ```yaml + --8<-- "examples/build_recipes/cicd/github-actions/deploy-s3.yml" + ``` + +=== "Build tool integration" + + ```yaml + --8<-- "examples/build_recipes/cicd/github-actions/deploy-build-tools.yml" + ``` + +## AWS CodeBuild + +**AWS CodeBuild** is a fully managed build service that compiles source code, runs tests, and produces deployment packages. It integrates seamlessly with other AWS services and provides consistent build environments with automatic scaling. + +=== "Basic CodeBuild Configuration" + + ```yaml + --8<-- "examples/build_recipes/cicd/codebuild/buildspec.yml" + ``` + +## Best Practices for CI/CD + +1. **Use Linux runners** (ubuntu-latest) to ensure Lambda compatibility +2. **Cache dependencies** to speed up builds (uv, poetry cache, pip cache) +3. **Run tests first** before building deployment packages +4. **Use matrix builds** to test multiple Python versions or configurations +5. **Implement proper secrets management** with GitHub Secrets or AWS Parameter Store +6. **Add deployment gates** for production environments +7. **Monitor deployment success** with CloudWatch metrics and alarms + +???+ tip "Performance Optimization" + - Use **uv** for fastest dependency installation in CI/CD + - **Cache virtual environments** between builds when possible + - **Parallelize builds** for multiple environments + - **Use container images** for complex dependencies or large packages diff --git a/docs/build_recipes/cross-platform.md b/docs/build_recipes/cross-platform.md new file mode 100644 index 00000000000..768ed93332a --- /dev/null +++ b/docs/build_recipes/cross-platform.md @@ -0,0 +1,238 @@ +--- +title: Cross-Platform Build Considerations +description: Handle architecture differences when building Lambda packages +--- + + + +Many modern Python packages include compiled extensions written in Rust or C/C++ for performance reasons. These compiled components are platform-specific and can cause deployment issues when building on different architectures. + +???+ warning "Architecture Mismatch Issues" + Building Lambda packages on macOS (ARM64/Intel) for deployment on AWS Lambda (Linux x86_64 or ARM64) will result in incompatible binary dependencies that cause import errors at runtime. + +## Common compiled libraries + +Taking into consideration Powertools for AWS dependencies and common Python packages, these libraries include compiled Rust/C components that require architecture-specific builds: + +| Library | Language | Components | Impact | Used in Powertools for AWS| +|---------|----------|------------|--------|-------------------| +| **pydantic** | Rust | Core validation engine | High - Core functionality affected | ✅ Core dependency | +| **aws-encryption-sdk** | C | Encryption/decryption | High - Data masking fails | ✅ Optional (datamasking extra) | +| **protobuf** | C++ | Protocol buffer serialization | High - Message parsing fails | ✅ Optional (kafka-consumer-protobuf) | +| **redis** | C | Redis client with hiredis | Medium - Falls back to pure Python | ✅ Optional (redis extra) | +| **valkey-glide** | Rust | High-performance Redis client | High - Client completely broken | ✅ Optional (valkey extra) | +| **orjson** | Rust | JSON serialization | Medium - Performance degradation | ❌ Not used (but common) | +| **uvloop** | C | Event loop implementation | Medium - Falls back to asyncio | ❌ Not used (but common) | +| **lxml** | C | XML/HTML processing | High - XML parsing fails | ❌ Not used (but common) | + +## Powertools extras dependencies and architecture + +Different Powertools for AWS extras dependencies have varying levels of architecture dependency: + +=== "Safe extras (pure python)" + + ```txt title="requirements.txt - Safe for any platform" + --8<-- "examples/build_recipes/build_multi_arch/requirements-safe-extras.txt" + ``` + +=== "Architecture-dependent extras" + + ```txt title="requirements.txt - Requires Linux builds" + --8<-- "examples/build_recipes/build_multi_arch/requirements-arch-dependent.txt" + ``` + +=== "All extras (mixed dependencies)" + + ```txt title="requirements.txt - Requires careful platform handling" + --8<-- "examples/build_recipes/build_multi_arch/requirements-all-extras.txt" + ``` + +???+ tip "Powertools for AWS build strategy" + 1. **Use `[all]` extra with Docker builds** for maximum compatibility + 2. **Use specific extras** if you want to avoid certain compiled dependencies + 3. **Test imports** after building to catch architecture mismatches early + +## Understanding Python wheels + +Python wheels are binary distribution packages that include compiled extensions. Lambda requires specific wheel types based on the target runtime architecture and GLIBC version. + +### Wheel naming convention + +Wheels follow a specific naming pattern that indicates compatibility: + +```txt +{package}-{version}-{python tag}-{abi tag}-{platform tag}.whl +``` + +**Example breakdown:** + +```txt +pydantic-2.5.0-cp311-cp311-linux_x86_64.whl +│ │ │ │ └─ Platform: Linux x86_64 +│ │ │ └─ ABI: CPython 3.11 +│ │ └─ Python: CPython 3.11 +│ └─ Version: 2.5.0 +└─ Package: pydantic +``` + +### Platform tags and Lambda compatibility + +| Platform Tag | Description | Lambda Compatibility | +|--------------|-------------|---------------------| +| `linux_x86_64` | Linux 64-bit Intel/AMD | ✅ Lambda x86_64 | +| `linux_aarch64` | Linux 64-bit ARM | ✅ Lambda arm64 | +| `macosx_*` | macOS (any version) | ❌ Not compatible | +| `win_*` | Windows (any version) | ❌ Not compatible | +| `any` | Pure Python, no compiled code | ✅ All platforms | + +### Source distributions vs wheels + +| Package Type | Extension | Compilation | Lambda Recommendation | +|--------------|-----------|-------------|----------------------| +| **Wheel** | `.whl` | Pre-compiled | ✅ Preferred - faster, consistent | +| **Source Distribution** | `.tar.gz` | Compile during install | ❌ Avoid - platform-dependent compilation | + +**Why wheels matter for Lambda:** + +* **Consistent builds** - Same binary across environments +* **Faster installs** - No compilation step required +* **Predictable dependencies** - Known system library requirements +* **Architecture safety** - Platform-specific binaries + +## Lambda runtime environments + +Lambda runtimes use specific Amazon Linux versions with fixed GLIBC versions. Packages built on development machines with newer GLIBC versions will fail at runtime with import errors. Each Python runtime version corresponds to a specific Amazon Linux base system that determines compatible system library versions. + +### Amazon Linux versions and GLIBC compatibility + +| Python Runtime | Base System | GLIBC Version | Architecture Support | Status | +|----------------|-------------|---------------|---------------------|---------| +| **python3.9** | Amazon Linux 2 | 2.26 | x86_64, arm64 | Supported | +| **python3.10** | Amazon Linux 2 | 2.26 | x86_64, arm64 | Supported | +| **python3.11** | Amazon Linux 2 | 2.26 | x86_64, arm64 | Supported | +| **python3.12** | Amazon Linux 2023 | 2.34 | x86_64, arm64 | Supported | +| **python3.13** | Amazon Linux 2023 | 2.34 | x86_64, arm64 | Supported | + +???+ warning "GLIBC Version Mismatch" + Compiled libraries built on systems with newer GLIBC versions will fail on Lambda runtimes with older GLIBC versions. Ubuntu 24.04 (GLIBC 2.39) and Ubuntu 22.04 (GLIBC 2.35) are incompatible with Lambda python3.11 and earlier (GLIBC 2.26). Always use `--platform` flags or Docker with Lambda base images. + +### Manylinux compatibility tags + +Python wheels use manylinux tags to indicate GLIBC compatibility. Understanding these tags helps you choose the right wheels for Lambda: + +| manylinux Tag | GLIBC Version | Lambda Compatibility | Recommendation | +|---------------|---------------|---------------------|----------------| +| **manylinux1** | 2.5 | ✅ All runtimes | Legacy, limited package support | +| **manylinux2010** | 2.12 | ✅ All runtimes | Good compatibility | +| **manylinux2014** | 2.17 | ✅ All runtimes | Recommended for most packages | +| **manylinux_2_17** | 2.17 | ✅ All runtimes | Modern standard | +| **manylinux_2_24** | 2.24 | ✅ All runtimes | Good for newer packages | +| **manylinux_2_28** | 2.28 | ✅ python3.12+, ❌ python3.11- | Use with caution | +| **manylinux_2_34** | 2.34 | ✅ python3.12+, ❌ python3.11- | AL2023 only | + +## Runtime-specific considerations + +### Amazon Linux 2 (python3.8 - python3.11) + +Amazon Linux 2 is based on RHEL 7 and uses an older GLIBC version (2.26). This provides broad compatibility but may limit access to newer compiled features. + +**Characteristics:** + +* **GLIBC 2.26** - Compatible with most manylinux wheels +* **OpenSSL 1.0.2** - Legacy TLS support + +**Best practices:** + +```bash +--8<-- "examples/build_recipes/build_multi_arch/build-al2.sh" +``` + +### Amazon Linux 2023 (python3.12+) + +Amazon Linux 2023 is a modern, minimal Linux distribution with updated system libraries and better security. + +**Characteristics:** + +* **GLIBC 2.34** - Supports newer compiled libraries +* **OpenSSL 3.0** - Latest TLS and cryptographic features +* **Smaller footprint** - Optimized for containers and serverless + +**Migration considerations:** + +```bash +--8<-- "examples/build_recipes/build_multi_arch/build-al2023.sh" +``` + +## Multi-platform build strategies + +=== "Docker-based Builds (Recommended)" + + Use AWS Lambda base images to ensure Linux x86_64 or ARM64 compatibility: + + === "Dockerfile" + + ```dockerfile + --8<-- "examples/build_recipes/build_multi_arch/Dockerfile.lambda" + ``` + + === "Build Script" + + ```bash + --8<-- "examples/build_recipes/build_multi_arch/build-multiplatform.sh" + ``` + +=== "Platform-specific pip install" + + Force installation of Linux-compatible wheels: + + === "Build Script" + + ```bash + --8<-- "examples/build_recipes/build_multi_arch/build-linux-wheels.sh" + ``` + +=== "GitHub Actions multi-arch" + + Use GitHub Actions with Linux runners for consistent builds: + + === "Workflow" + + ```yaml + --8<-- "examples/build_recipes/build_multi_arch/lambda-build.yml" + ``` + +## Debugging compatibility issues + +When you encounter runtime errors related to compiled dependencies, use these techniques to diagnose and fix the issues: + +### Common error patterns + +=== "GLIBC version errors" + + ```bash + --8<-- "examples/build_recipes/build_multi_arch/debug-glibc.sh" + ``` + +=== "Architecture mismatch" + + ```bash + --8<-- "examples/build_recipes/build_multi_arch/debug-arch-mismatch.sh" + ``` + +=== "Missing system libraries" + + ```bash + --8<-- "examples/build_recipes/build_multi_arch/debug-missing-libs.sh" + ``` + +## Best practices for cross-platform builds + +???+ tip "Development Workflow" + Develop locally on your preferred platform, but always build deployment packages in a Linux environment or Docker container to ensure compatibility. + +1. **Always build on Linux** for Lambda deployments, or use Docker with Lambda base images +2. **Use `--platform` flags** when installing with pip to force Linux-compatible wheels +3. **Test imports** in your build environment before deployment +4. **Pin dependency versions** to ensure reproducible builds across platforms +5. **Use CI/CD with Linux runners** to avoid local architecture issues +6. **Consider Lambda container images** for complex dependency scenarios diff --git a/docs/build_recipes/getting-started.md b/docs/build_recipes/getting-started.md new file mode 100644 index 00000000000..a03cd772b4e --- /dev/null +++ b/docs/build_recipes/getting-started.md @@ -0,0 +1,30 @@ +--- +title: Getting Started +description: Prerequisites and setup for building Lambda functions with Powertools +--- + + + +## Prerequisites + +Before using any of these recipes, ensure you have: + +* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html){target="_blank"} configured +* Python 3.10+ installed +* Your preferred build tool installed (see individual recipes) + +## Choosing the right tool + +Each build tool has its strengths and is optimized for different use cases. Consider your project complexity, team preferences, and deployment requirements when selecting the best approach. + +| Tool | Best for | Considerations | +| --------------------- | --------------------------------- | ------------------------------------------- | +| **[pip](build-with-pip.md)** | Simple projects, CI/CD | Lightweight, universal | +| **[poetry](build-with-poetry.md)** | Modern Python projects | Excellent dependency management, lock files | +| **[uv](build-with-uv.md)** | Fast builds, performance-critical | Extremely fast, Rust-based | +| **[pants](build-with-pants.md)** | Monorepos, complex projects | Advanced build system, incremental builds | +| **[SAM](build-with-sam.md)** | AWS-native deployments | Integrated with AWS, local testing | +| **[CDK](build-with-cdk.md)** | Infrastructure as code | Programmatic infrastructure, type safety | + +???+ tip + All examples in this guide are available in the [project repository](https://github.com/aws-powertools/powertools-lambda-python/tree/develop/examples/build_recipes){target="_blank"}. diff --git a/docs/build_recipes/index.md b/docs/build_recipes/index.md new file mode 100644 index 00000000000..5710c997c19 --- /dev/null +++ b/docs/build_recipes/index.md @@ -0,0 +1,57 @@ +--- +title: Build Recipes +description: Lambda function packaging recipes with Powertools for AWS +--- + + + +As the Python ecosystem continues to evolve with new package managers, build tools, and dependency resolution strategies, choosing the right approach for Lambda deployments has become increasingly complex. Modern Python applications often involve compiled extensions, platform-specific dependencies, and sophisticated toolchains that require careful consideration for serverless environments. + +This guide provides practical recipes for packaging Lambda functions with Powertools for AWS Lambda (Python) using different build tools and dependency managers. + +## Key benefits + +* **Optimized packaging** - Reduce deployment package size and cold start times +* **Dependency management** - Handle complex dependency trees efficiently +* **Build reproducibility** - Consistent builds across environments +* **Layer optimization** - Leverage Lambda Layers for better performance +* **Multi-tool support** - Choose the right tool for your workflow + +## Terminology + +Understanding these key terms will help you navigate the build recipes more effectively: + +| Term | Definition | +|------|------------| +| **Deployment Package** | A ZIP archive or container image containing your Lambda function code and all its dependencies, ready for deployment to AWS Lambda | +| **Lambda Layer** | A ZIP archive containing libraries, custom runtimes, or other function dependencies that can be shared across multiple Lambda functions | +| **Build Tool** | Software that automates the process of compiling, packaging, and preparing your code for deployment (e.g., pip, poetry, uv, pants) | +| **Dependency Manager** | Tool responsible for resolving, downloading, and managing external libraries your project depends on | +| **Lock File** | A file that records the exact versions of all dependencies used in your project, ensuring reproducible builds (e.g., poetry.lock, uv.lock) | +| **Cold Start** | The initialization time when AWS Lambda creates a new execution environment for your function, including loading your deployment package | +| **SAM (Serverless Application Model)** | AWS framework for building serverless applications, providing templates and CLI tools for deploying Lambda functions and related resources | +| **CDK (Cloud Development Kit)** | AWS framework for defining cloud infrastructure using familiar programming languages, enabling infrastructure as code for Lambda deployments | + +## Guide sections + +This guide is organized into focused sections to help you find exactly what you need: + +### 📚 Fundamentals + +* **[Getting started](getting-started.md)** - Prerequisites, tool selection, and basic setup +* **[Cross-platform builds](cross-platform.md)** - Handle architecture differences and compiled dependencies + +### 🔧 Build tools + +* **[Build with pip](build-with-pip.md)** - Simple, universal package management +* **[Build with Poetry](build-with-poetry.md)** - Modern dependency management with lock files +* **[Build with uv](build-with-uv.md)** - Extremely fast Rust-based package manager +* **[Build with SAM](build-with-sam.md)** - AWS Serverless Application Model integration +* **[Build with CDK](build-with-cdk.md)** - Infrastructure as code with type safety +* **[Build with Pants](build-with-pants.md)** - Advanced build system for monorepos + +### ⚡ Advanced topics + +* **[Performance optimization](performance-optimization.md)** - Reduce cold starts and package size +* **[CI/CD integration](cicd-integration.md)** - Automate builds with GitHub Actions and CodeBuild +* **[Troubleshooting](troubleshooting.md)** - Common issues and solutions diff --git a/docs/build_recipes/performance-optimization.md b/docs/build_recipes/performance-optimization.md new file mode 100644 index 00000000000..22431999d47 --- /dev/null +++ b/docs/build_recipes/performance-optimization.md @@ -0,0 +1,34 @@ +--- +title: Performance Optimization +description: Optimize Lambda functions for better performance and reduced costs +--- + + + +Optimize your Lambda functions for better performance, reduced cold start times, and lower costs. These techniques help minimize package size, improve startup speed, and reduce memory usage. + +## Reduce cold start times + +1. **Minimize package size** by excluding unnecessary files +2. **Use compiled dependencies** when possible +3. **Leverage Lambda SnapStart** or **Provisioned concurrency** when possible + +## Build optimization + +=== "Exclude unnecessary files" + + ```bash + --8<-- "examples/build_recipes/build_optimization/optimize-package.sh" + ``` + +=== "Layer optimization" + + ```bash + --8<-- "examples/build_recipes/build_optimization/optimize-layer.sh" + ``` + +=== "Advanced optimization with debug symbol removal" + + ```bash + --8<-- "examples/build_recipes/build_optimization/optimize-advanced.sh" + ``` diff --git a/docs/build_recipes/troubleshooting.md b/docs/build_recipes/troubleshooting.md new file mode 100644 index 00000000000..0972ef88aee --- /dev/null +++ b/docs/build_recipes/troubleshooting.md @@ -0,0 +1,142 @@ +--- +title: Troubleshooting +description: Common issues and solutions when building Lambda packages +--- + + + +## Common issues and solutions + +### Package size issues + +???+ warning "Lambda deployment package too large (>50MB unzipped)" + **Symptoms:** + - `RequestEntityTooLargeException` during deployment + - Slow cold starts + - High memory usage + + **Solutions:** + ```bash + # 1. Use Lambda Layers for heavy dependencies + pip install aws-lambda-powertools[all] -t layers/powertools/python/ + + # 2. Remove unnecessary files + find build/ -name "*.pyc" -delete + find build/ -name "__pycache__" -type d -exec rm -rf {} + + find build/ -name "tests" -type d -exec rm -rf {} + + + # 3. Strip debug symbols from compiled libraries + find build/ -name "*.so" -exec strip --strip-debug {} \; + + # 4. Use container images for very large packages + # Deploy as container image instead of ZIP + ``` + +### Import and runtime errors + +???+ error "ModuleNotFoundError or ImportError" + **Symptoms:** + - `ModuleNotFoundError: No module named 'aws_lambda_powertools'` + - Function fails at runtime with import errors + + **Solutions:** + ```bash + # 1. Verify dependencies are in the package + unzip -l lambda-package.zip | grep powertools + + # 2. Check Python path in Lambda + python -c "import sys; print(sys.path)" + + # 3. Ensure platform compatibility + pip install --platform linux_x86_64 --only-binary=:all: aws-lambda-powertools[all] + + # 4. Test imports locally + cd build && python -c "from aws_lambda_powertools import Logger; print('OK')" + ``` + +???+ error "Architecture mismatch errors" + **Symptoms:** + - `ImportError: /lib64/libc.so.6: version GLIBC_2.XX not found` + - Compiled extensions fail to load + + **Solutions:** + ```bash + # Use Docker with Lambda base image + docker run --rm -v "$PWD":/var/task public.ecr.aws/lambda/python:3.13 \ + pip install aws-lambda-powertools[all] -t /var/task/ + + # Or force Linux-compatible wheels + pip install --platform linux_x86_64 --implementation cp \ + --python-version 3.13 --only-binary=:all: aws-lambda-powertools[all] + ``` + +### Performance issues + +???+ warning "Slow cold starts" + **Symptoms:** + - High initialization duration in CloudWatch logs + - Timeouts on first invocation + + **Solutions:** + ```bash + # 1. Optimize package size (see above) + + # 2. Use public Powertools for AWS layer + # Layer ARN: arn:aws:lambda:region:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-x86_64:1 + + # 3. Enable provisioned concurrency for critical functions + aws lambda put-provisioned-concurrency-config \ + --function-name my-function \ + --provisioned-concurrency-config ProvisionedConcurrencyCount=10 + + # 4. Minimize imports in handler + # Import only what you need, avoid heavy imports at module level + ``` + +### Build and deployment issues + +???+ error "Build inconsistencies across environments" + **Symptoms:** + - Works locally but fails in CI/CD + - Different behavior between team members + + **Solutions:** + ```bash + # 1. Use lock files for reproducible builds + # Poetry: poetry.lock + # uv: uv.lock + # pip: requirements.txt with pinned versions + + # 2. Use Docker for consistent build environment + docker run --rm -v "$PWD":/app -w /app python:3.13-slim \ + bash -c "pip install -r requirements.txt -t build/" + + # 3. Pin all tool versions + pip==24.0 + poetry==1.8.0 + uv==0.1.0 + + # 4. Use same Python version everywhere + python-version: '3.13' # In CI/CD + python = "^3.13" # In pyproject.toml + ``` + +???+ error "Layer compatibility issues" + **Symptoms:** + - Layer not found or incompatible runtime + - Version conflicts between layer and function dependencies + + **Solutions:** + ```bash + # 1. Use correct layer ARN for your region and Python version + # Check: https://docs.powertools.aws.dev/lambda/python/latest/#lambda-layer + + # 2. Verify layer compatibility + aws lambda get-layer-version \ + --layer-name AWSLambdaPowertoolsPythonV3-python313-x86_64 \ + --version-number 1 + + # 3. Avoid version conflicts + # Don't include Powertools for AWS in deployment package if using layer + pip install pydantic requests -t build/ # Exclude powertools + ``` diff --git a/examples/build_recipes/build_multi_arch/Dockerfile.lambda b/examples/build_recipes/build_multi_arch/Dockerfile.lambda new file mode 100644 index 00000000000..629555f21d2 --- /dev/null +++ b/examples/build_recipes/build_multi_arch/Dockerfile.lambda @@ -0,0 +1,16 @@ +#Public Lambda image +FROM public.ecr.aws/lambda/python@sha256:7e7f098baa11a527fbe59f33f4ed032a36b6e87b22ea73da1175522095885f74 + +# Set workdir file +WORKDIR /tmp/app + +# Copy requirements first for better caching +COPY requirements.txt . + +# Install dependencies in Lambda-compatible environment +RUN pip install -r requirements.txt + +# Copy application code +COPY . . + +CMD ["app.lambda_handler"] diff --git a/examples/build_recipes/build_multi_arch/build-al2.sh b/examples/build_recipes/build_multi_arch/build-al2.sh new file mode 100644 index 00000000000..ded510e39e4 --- /dev/null +++ b/examples/build_recipes/build_multi_arch/build-al2.sh @@ -0,0 +1,5 @@ +#!/bin/bash +# Use Amazon Linux 2 base image for builds +docker run --rm -v "$PWD":/var/task \ + public.ecr.aws/lambda/python:3.11 \ + pip install -r requirements.txt -t /var/task/ diff --git a/examples/build_recipes/build_multi_arch/build-al2023.sh b/examples/build_recipes/build_multi_arch/build-al2023.sh new file mode 100644 index 00000000000..3347a5bdef7 --- /dev/null +++ b/examples/build_recipes/build_multi_arch/build-al2023.sh @@ -0,0 +1,9 @@ +#!/bin/bash +# Some packages may require rebuilding for AL2023 +# Check for GLIBC symbol errors in logs: +# ImportError: /lib64/libc.so.6: version `GLIBC_2.34' not found + +# Use AL2023 base image for python3.12+ +docker run --rm -v "$PWD":/var/task \ + public.ecr.aws/lambda/python:3.12 \ + pip install -r requirements.txt -t /var/task/ diff --git a/examples/build_recipes/build_multi_arch/build-linux-wheels.sh b/examples/build_recipes/build_multi_arch/build-linux-wheels.sh new file mode 100644 index 00000000000..1b0c28a4706 --- /dev/null +++ b/examples/build_recipes/build_multi_arch/build-linux-wheels.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +# Create build directory +mkdir -p build/ + +# Install Linux-compatible wheels +pip install \ + --platform manylinux2014_x86_64 \ + --target build/ \ + --implementation cp \ + --python-version 3.13 \ + --only-binary=:all: \ + --upgrade \ + --abi cp313 \ + -r requirements.txt + +# Copy application code +cp -r src/* build/ + +# Create deployment package +cd build && zip -r ../lambda-linux.zip . && cd .. + +echo "✅ Linux-compatible package created" diff --git a/examples/build_recipes/build_multi_arch/build-multiplatform.sh b/examples/build_recipes/build_multi_arch/build-multiplatform.sh new file mode 100644 index 00000000000..87607f89bd5 --- /dev/null +++ b/examples/build_recipes/build_multi_arch/build-multiplatform.sh @@ -0,0 +1,14 @@ + #!/bin/bash + +# Build using Lambda-compatible environment +docker build -f Dockerfile.lambda -t lambda-build . + +# Extract built packages +docker create --name temp-container lambda-build +docker cp temp-container:/var/task ./build +docker rm temp-container + +# Create deployment package +cd build && zip -r ../lambda-multiplatform.zip . && cd .. + +echo "✅ Multi-platform compatible package created" diff --git a/examples/build_recipes/build_multi_arch/debug-arch-mismatch.sh b/examples/build_recipes/build_multi_arch/debug-arch-mismatch.sh new file mode 100644 index 00000000000..574421db95b --- /dev/null +++ b/examples/build_recipes/build_multi_arch/debug-arch-mismatch.sh @@ -0,0 +1,12 @@ +#!/bin/bash +# Error message: +# ImportError: cannot import name '_speedups' from 'pydantic' + +# Check library architecture +file /opt/python/lib/python3.11/site-packages/pydantic/_internal/_pydantic_core.so + +# Expected output for Lambda x86_64: +# ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked + +# Solution: Force correct platform +pip install --platform linux_x86_64 --force-reinstall pydantic -t build/ diff --git a/examples/build_recipes/build_multi_arch/debug-glibc.sh b/examples/build_recipes/build_multi_arch/debug-glibc.sh new file mode 100644 index 00000000000..ec4caae76f7 --- /dev/null +++ b/examples/build_recipes/build_multi_arch/debug-glibc.sh @@ -0,0 +1,13 @@ +#!/bin/bash +# Error message: +# ImportError: /lib64/libc.so.6: version `GLIBC_2.34' not found + +# Check GLIBC version in Lambda runtime +ldd --version + +# Check required GLIBC symbols in a library +objdump -T /opt/python/lib/python3.11/site-packages/pydantic/_internal/_pydantic_core.so | grep GLIBC + +# Solution: Rebuild with compatible base image +docker run --rm -v "$PWD":/var/task public.ecr.aws/lambda/python:3.11 \ + pip install --force-reinstall pydantic -t /var/task/ diff --git a/examples/build_recipes/build_multi_arch/debug-missing-libs.sh b/examples/build_recipes/build_multi_arch/debug-missing-libs.sh new file mode 100644 index 00000000000..140e1b289be --- /dev/null +++ b/examples/build_recipes/build_multi_arch/debug-missing-libs.sh @@ -0,0 +1,9 @@ +#!/bin/bash +# Error message: +# ImportError: libffi.so.6: cannot open shared object file + +# Check library dependencies +ldd /opt/python/lib/python3.11/site-packages/some_package/_extension.so + +# Solution: Use Lambda base image with system dependencies +# Or switch to pure Python alternatives diff --git a/examples/build_recipes/build_multi_arch/lambda-build.yml b/examples/build_recipes/build_multi_arch/lambda-build.yml new file mode 100644 index 00000000000..432c1caf18a --- /dev/null +++ b/examples/build_recipes/build_multi_arch/lambda-build.yml @@ -0,0 +1,68 @@ +name: Build Lambda Package + +on: + push: + branches: [main] + +jobs: + build: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Install dependencies with Lambda-compatible wheels + run: | + # Force Linux x86_64 wheels compatible with Lambda GLIBC 2.34 + pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build/ \ + -r requirements.txt + + - name: Copy application code + run: cp -r src/* build/ + + - name: Create deployment package + run: | + cd build && zip -r ../lambda-deployment.zip . && cd .. + + - name: Upload artifact + uses: actions/upload-artifact@v4 + with: + name: lambda-package + path: lambda-deployment.zip + + build-arm64: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Install dependencies for ARM64 Lambda + run: | + # Force Linux ARM64 wheels compatible with Lambda + pip install --platform manylinux2014_aarch64 --only-binary=:all: \ + --python-version 3.13 --target build/ \ + -r requirements.txt + + - name: Copy application code + run: cp -r src/* build/ + + - name: Create deployment package + run: | + cd build && zip -r ../lambda-deployment-arm64.zip . && cd .. + + - name: Upload artifact + uses: actions/upload-artifact@v4 + with: + name: lambda-package-arm64 + path: lambda-deployment-arm64.zip diff --git a/examples/build_recipes/build_multi_arch/requirements-all-extras.txt b/examples/build_recipes/build_multi_arch/requirements-all-extras.txt new file mode 100644 index 00000000000..b24b16471ac --- /dev/null +++ b/examples/build_recipes/build_multi_arch/requirements-all-extras.txt @@ -0,0 +1,6 @@ +# The 'all' extra includes both safe and architecture-dependent packages +aws-lambda-powertools[all]==3.18.0 + +# This is equivalent to: +# pydantic, pydantic-settings, aws-xray-sdk, fastjsonschema, +# aws-encryption-sdk, jsonpath-ng diff --git a/examples/build_recipes/build_multi_arch/requirements-arch-dependent.txt b/examples/build_recipes/build_multi_arch/requirements-arch-dependent.txt new file mode 100644 index 00000000000..d612dff860d --- /dev/null +++ b/examples/build_recipes/build_multi_arch/requirements-arch-dependent.txt @@ -0,0 +1,8 @@ +# These extras include compiled dependencies +aws-lambda-powertools[parser]==3.18.0 # pydantic (Rust) +aws-lambda-powertools[validation]==3.18.0 # fastjsonschema (C) +aws-lambda-powertools[datamasking]==3.18.0 # aws-encryption-sdk (C) +aws-lambda-powertools[redis]==3.18.0 # redis with hiredis (C) +aws-lambda-powertools[valkey]==3.18.0 # valkey-glide (Rust) +aws-lambda-powertools[kafka-consumer-avro]==3.18.0 # avro (C) +aws-lambda-powertools[kafka-consumer-protobuf]==3.18.0 # protobuf (C++) diff --git a/examples/build_recipes/build_multi_arch/requirements-safe-extras.txt b/examples/build_recipes/build_multi_arch/requirements-safe-extras.txt new file mode 100644 index 00000000000..da0ab40174d --- /dev/null +++ b/examples/build_recipes/build_multi_arch/requirements-safe-extras.txt @@ -0,0 +1,3 @@ +# These extras have minimal or no compiled dependencies +aws-lambda-powertools[tracer]==3.18.0 # aws-xray-sdk (mostly pure Python) +aws-lambda-powertools[aws-sdk]==3.18.0 # boto3 (pure Python) diff --git a/examples/build_recipes/build_optimization/optimize-advanced.sh b/examples/build_recipes/build_optimization/optimize-advanced.sh new file mode 100644 index 00000000000..b27b466166f --- /dev/null +++ b/examples/build_recipes/build_optimization/optimize-advanced.sh @@ -0,0 +1,26 @@ +#!/bin/bash + +# Remove unnecessary files +find build/ -name "*.pyc" -delete +find build/ -name "__pycache__" -type d -exec rm -rf {} + +find build/ -name "*.dist-info" -type d -exec rm -rf {} + +find build/ -name "tests" -type d -exec rm -rf {} + +find build/ -name "test_*" -delete + +# Remove debug symbols from compiled extensions +find build/ -name "*.so" -exec strip --strip-debug {} \; 2>/dev/null || true +find build/ -name "*.so.*" -exec strip --strip-debug {} \; 2>/dev/null || true + +# Remove additional bloat from common packages +rm -rf build/*/site-packages/*/tests/ +rm -rf build/*/site-packages/*/test/ +rm -rf build/*/site-packages/*/.git/ +rm -rf build/*/site-packages/*/docs/ +rm -rf build/*/site-packages/*/examples/ +rm -rf build/*/site-packages/*/*.md +rm -rf build/*/site-packages/*/*.rst +rm -rf build/*/site-packages/*/*.txt + +# Calculate size reduction +echo "📊 Package optimization completed" +du -sh build/ 2>/dev/null || echo "✅ Advanced optimization applied" diff --git a/examples/build_recipes/build_optimization/optimize-layer.sh b/examples/build_recipes/build_optimization/optimize-layer.sh new file mode 100644 index 00000000000..3b3fc6ded95 --- /dev/null +++ b/examples/build_recipes/build_optimization/optimize-layer.sh @@ -0,0 +1,18 @@ +#!/bin/bash + +# Create optimized layer structure +mkdir -p layer/python/ + +# Install only production dependencies +pip install aws-lambda-powertools[all] -t layer/python/ --no-deps +pip install pydantic -t layer/python/ --no-deps + +# Remove unnecessary files from layer +find layer/ -name "*.pyc" -delete +find layer/ -name "__pycache__" -type d -exec rm -rf {} + +find layer/ -name "tests" -type d -exec rm -rf {} + + +# Create layer zip +cd layer && zip -r ../optimized-layer.zip . && cd .. + +echo "✅ Optimized layer created: optimized-layer.zip" diff --git a/examples/build_recipes/build_optimization/optimize-package.sh b/examples/build_recipes/build_optimization/optimize-package.sh new file mode 100644 index 00000000000..d0521c3f532 --- /dev/null +++ b/examples/build_recipes/build_optimization/optimize-package.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +# Remove unnecessary files to reduce package size +find build/ -name "*.pyc" -delete +find build/ -name "__pycache__" -type d -exec rm -rf {} + +find build/ -name "*.dist-info" -type d -exec rm -rf {} + +find build/ -name "tests" -type d -exec rm -rf {} + +find build/ -name "test_*" -delete + +# Remove documentation and examples +find build/ -name "docs" -type d -exec rm -rf {} + +find build/ -name "examples" -type d -exec rm -rf {} + + +echo "✅ Package optimized" diff --git a/examples/build_recipes/cdk/basic/app.py b/examples/build_recipes/cdk/basic/app.py new file mode 100644 index 00000000000..3ce92f91f70 --- /dev/null +++ b/examples/build_recipes/cdk/basic/app.py @@ -0,0 +1,74 @@ +#!/usr/bin/env python3 +import aws_cdk as cdk +from aws_cdk import ( + Duration, + Stack, +) +from aws_cdk import ( + aws_apigateway as apigateway, +) +from aws_cdk import ( + aws_lambda as _lambda, +) +from aws_cdk import ( + aws_logs as logs, +) +from constructs import Construct + + +class PowertoolsLambdaStack(Stack): + def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: + super().__init__(scope, construct_id, **kwargs) + + # Use public Powertools layer + powertools_layer = _lambda.LayerVersion.from_layer_version_arn( + self, + "PowertoolsLayer", + layer_version_arn="arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-x86_64:1", + ) + + # Lambda Function + api_function = _lambda.Function( + self, + "ApiFunction", + runtime=_lambda.Runtime.PYTHON_3_13, + handler="lambda_function.lambda_handler", + code=_lambda.Code.from_asset("src"), + layers=[powertools_layer], + timeout=Duration.seconds(30), + memory_size=512, + environment={ + "POWERTOOLS_SERVICE_NAME": "api-service", + "POWERTOOLS_METRICS_NAMESPACE": "MyApp", + "POWERTOOLS_LOG_LEVEL": "INFO", + }, + log_retention=logs.RetentionDays.ONE_WEEK, + ) + + # API Gateway + api = apigateway.RestApi( + self, + "ApiGateway", + rest_api_name="Powertools API", + description="API powered by Lambda with Powertools", + ) + + # API Integration + integration = apigateway.LambdaIntegration(api_function) + api.root.add_proxy( + default_integration=integration, + any_method=True, + ) + + # Outputs + cdk.CfnOutput( + self, + "ApiUrl", + value=api.url, + description="API Gateway URL", + ) + + +app = cdk.App() +PowertoolsLambdaStack(app, "PowertoolsLambdaStack") +app.synth() diff --git a/examples/build_recipes/cdk/basic/build-cdk.sh b/examples/build_recipes/cdk/basic/build-cdk.sh new file mode 100644 index 00000000000..df689d7ba2f --- /dev/null +++ b/examples/build_recipes/cdk/basic/build-cdk.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +echo "🏗️ Building CDK application..." + +# Install CDK dependencies +pip install -r requirements.txt + +# Bootstrap CDK (first time only) +# cdk bootstrap + +# Deploy stack +cdk deploy --require-approval never + +echo "✅ CDK application deployed successfully" diff --git a/examples/build_recipes/cdk/basic/cdk-commands.sh b/examples/build_recipes/cdk/basic/cdk-commands.sh new file mode 100644 index 00000000000..e975ad5bc1e --- /dev/null +++ b/examples/build_recipes/cdk/basic/cdk-commands.sh @@ -0,0 +1,21 @@ +#!/bin/bash +# Install Python dependencies +pip install -r requirements.txt + +# Synthesize CloudFormation template +cdk synth + +# Deploy stack +cdk deploy + +# Deploy specific stack +cdk deploy MyLambdaStack + +# Destroy stack +cdk destroy + +# List all stacks +cdk list + +# Compare deployed stack with current state +cdk diff diff --git a/examples/build_recipes/cdk/basic/cdk.json b/examples/build_recipes/cdk/basic/cdk.json new file mode 100644 index 00000000000..1e8e4ddaaac --- /dev/null +++ b/examples/build_recipes/cdk/basic/cdk.json @@ -0,0 +1,37 @@ +{ + "app": "python app.py", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "requirements*.txt", + "source.bat", + "**/__pycache__", + "**/.venv" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": ["aws", "aws-cn"], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/core:validateSnapshotRemovalPolicy": true, + "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, + "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, + "@aws-cdk/core:enablePartitionLiterals": true, + "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/core:validateSnapshotRemovalPolicy": true, + "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true, + "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, + "@aws-cdk/aws-route53-patters:useCertificate": true, + "@aws-cdk/customresources:installLatestAwsSdkDefault": false + } +} diff --git a/examples/build_recipes/cdk/basic/requirements.txt b/examples/build_recipes/cdk/basic/requirements.txt new file mode 100644 index 00000000000..e8640c9f492 --- /dev/null +++ b/examples/build_recipes/cdk/basic/requirements.txt @@ -0,0 +1,2 @@ +aws-cdk-lib>=2.100.0 +constructs>=10.0.0 diff --git a/examples/build_recipes/cdk/basic/setup-cdk.sh b/examples/build_recipes/cdk/basic/setup-cdk.sh new file mode 100644 index 00000000000..0940f9cd562 --- /dev/null +++ b/examples/build_recipes/cdk/basic/setup-cdk.sh @@ -0,0 +1,9 @@ +#!/bin/bash +# Install AWS CDK CLI +npm install -g aws-cdk + +# Verify installation +cdk --version + +# Bootstrap CDK in your AWS account (one-time setup) +cdk bootstrap aws://ACCOUNT-ID/REGION diff --git a/examples/build_recipes/cdk/basic/src/lambda_function.py b/examples/build_recipes/cdk/basic/src/lambda_function.py new file mode 100644 index 00000000000..025b8ef6fcd --- /dev/null +++ b/examples/build_recipes/cdk/basic/src/lambda_function.py @@ -0,0 +1,27 @@ +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.logging import correlation_paths +from aws_lambda_powertools.metrics import MetricUnit + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + + +@app.get("/health") +def health_check(): + return {"status": "healthy", "service": "powertools-cdk"} + + +@app.get("/metrics") +def get_metrics(): + metrics.add_metric(name="MetricsEndpointCalled", unit=MetricUnit.Count, value=1) + return {"message": "Metrics recorded"} + + +@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST) +@tracer.capture_lambda_handler +@metrics.log_metrics(capture_cold_start_metric=True) +def lambda_handler(event, context): + return app.resolve(event, context) diff --git a/examples/build_recipes/cdk/multi-stack/app_multi_stack.py b/examples/build_recipes/cdk/multi-stack/app_multi_stack.py new file mode 100644 index 00000000000..be2c8077e66 --- /dev/null +++ b/examples/build_recipes/cdk/multi-stack/app_multi_stack.py @@ -0,0 +1,21 @@ +#!/usr/bin/env python3 +import aws_cdk as cdk +from stacks.powertools_cdk_stack import PowertoolsStack + +app = cdk.App() + +# Get environment from context or default to dev +environment = app.node.try_get_context("environment") or "dev" + +# Create stack for the specified environment +PowertoolsStack( + app, + f"PowertoolsStack-{environment}", + environment=environment, + env=cdk.Environment( + account=app.node.try_get_context("account"), + region=app.node.try_get_context("region") or "us-east-1", + ), +) + +app.synth() diff --git a/examples/build_recipes/cdk/multi-stack/cdk.json b/examples/build_recipes/cdk/multi-stack/cdk.json new file mode 100644 index 00000000000..4e34e02f383 --- /dev/null +++ b/examples/build_recipes/cdk/multi-stack/cdk.json @@ -0,0 +1,37 @@ +{ + "app": "python app_multi_stack.py", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "requirements*.txt", + "source.bat", + "**/__pycache__", + "**/.venv" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": ["aws", "aws-cn"], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/core:validateSnapshotRemovalPolicy": true, + "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, + "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, + "@aws-cdk/core:enablePartitionLiterals": true, + "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/core:validateSnapshotRemovalPolicy": true, + "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true, + "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, + "@aws-cdk/aws-route53-patters:useCertificate": true, + "@aws-cdk/customresources:installLatestAwsSdkDefault": false + } +} diff --git a/examples/build_recipes/cdk/multi-stack/deploy-environments.sh b/examples/build_recipes/cdk/multi-stack/deploy-environments.sh new file mode 100644 index 00000000000..8adc8779c49 --- /dev/null +++ b/examples/build_recipes/cdk/multi-stack/deploy-environments.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +# Deploy to different environments +environments=("dev" "staging" "prod") + +for env in "${environments[@]}"; do + echo "🚀 Deploying to $env environment..." + + cdk deploy PowertoolsStack-$env \ + --context environment=$env \ + --require-approval never + + echo "✅ $env deployment completed" +done diff --git a/examples/build_recipes/cdk/multi-stack/src/app/api.py b/examples/build_recipes/cdk/multi-stack/src/app/api.py new file mode 100644 index 00000000000..63d3daffb9c --- /dev/null +++ b/examples/build_recipes/cdk/multi-stack/src/app/api.py @@ -0,0 +1,49 @@ +import os + +import boto3 + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.logging import correlation_paths +from aws_lambda_powertools.metrics import MetricUnit + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + +# Initialize AWS clients +dynamodb = boto3.resource("dynamodb") +sqs = boto3.client("sqs") + +table = dynamodb.Table(os.environ["TABLE_NAME"]) +queue_url = os.environ["QUEUE_URL"] + + +@app.get("/health") +def health_check(): + return {"status": "healthy", "service": "powertools-cdk-api"} + + +@app.post("/tasks") +@tracer.capture_method +def create_task(): + task_data = app.current_event.json_body + + # Store in DynamoDB + table.put_item(Item={"pk": task_data["task_id"], "task_type": task_data["task_type"], "status": "pending"}) + + # Send to SQS for processing + sqs.send_message(QueueUrl=queue_url, MessageBody=app.current_event.body) + + metrics.add_metric(name="TaskCreated", unit=MetricUnit.Count, value=1) + logger.info("Task created", extra={"task_id": task_data["task_id"]}) + + return {"message": "Task created successfully", "task_id": task_data["task_id"]} + + +@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST) +@tracer.capture_lambda_handler +@metrics.log_metrics(capture_cold_start_metric=True) +def lambda_handler(event, context): + return app.resolve(event, context) diff --git a/examples/build_recipes/cdk/multi-stack/src/worker/__init__.py b/examples/build_recipes/cdk/multi-stack/src/worker/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/examples/build_recipes/cdk/multi-stack/src/worker/worker.py b/examples/build_recipes/cdk/multi-stack/src/worker/worker.py new file mode 100644 index 00000000000..99157c3b078 --- /dev/null +++ b/examples/build_recipes/cdk/multi-stack/src/worker/worker.py @@ -0,0 +1,82 @@ +from __future__ import annotations + +import json +import os +from typing import Any + +import boto3 + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, process_partial_response +from aws_lambda_powertools.utilities.typing import LambdaContext + +logger = Logger() +tracer = Tracer() +metrics = Metrics() + +# Initialize batch processor for SQS +processor = BatchProcessor(event_type=EventType.SQS) + +# Initialize AWS clients +dynamodb = boto3.resource("dynamodb") +table = dynamodb.Table(os.environ["TABLE_NAME"]) + + +@tracer.capture_method +def record_handler(record): + """Process individual SQS record""" + try: + # Parse message + message_data = json.loads(record.body) + task_id = message_data["task_id"] + task_type = message_data["task_type"] + + logger.info("Processing task", extra={"task_id": task_id, "task_type": task_type}) + + # Update task status in DynamoDB + table.update_item( + Key={"pk": task_id}, + UpdateExpression="SET #status = :status", + ExpressionAttributeNames={"#status": "status"}, + ExpressionAttributeValues={":status": "processing"}, + ) + + # Simulate work based on task type + if task_type == "email": + logger.info("Sending email", extra={"task_id": task_id}) + elif task_type == "report": + logger.info("Generating report", extra={"task_id": task_id}) + else: + logger.warning("Unknown task type", extra={"task_type": task_type}) + + # Mark as completed + table.update_item( + Key={"pk": task_id}, + UpdateExpression="SET #status = :status", + ExpressionAttributeNames={"#status": "status"}, + ExpressionAttributeValues={":status": "completed"}, + ) + + metrics.add_metric(name="TaskProcessed", unit="Count", value=1) + metrics.add_metadata(key="task_type", value=task_type) + + return {"status": "success", "task_id": task_id} + + except Exception as e: + logger.error("Task processing failed", extra={"error": str(e)}) + metrics.add_metric(name="TaskFailed", unit="Count", value=1) + raise + + +@logger.inject_lambda_context +@tracer.capture_lambda_handler +@metrics.log_metrics +def lambda_handler(event: dict[str, Any], context: LambdaContext): + """Process SQS messages using BatchProcessor""" + + return process_partial_response( + event=event, + record_handler=record_handler, + processor=processor, + context=context, + ) diff --git a/examples/build_recipes/cdk/multi-stack/stacks/__init__.py b/examples/build_recipes/cdk/multi-stack/stacks/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/examples/build_recipes/cdk/multi-stack/stacks/powertools_cdk_stack.py b/examples/build_recipes/cdk/multi-stack/stacks/powertools_cdk_stack.py new file mode 100644 index 00000000000..c6666a834e4 --- /dev/null +++ b/examples/build_recipes/cdk/multi-stack/stacks/powertools_cdk_stack.py @@ -0,0 +1,144 @@ +from aws_cdk import ( + Duration, + RemovalPolicy, + Stack, +) +from aws_cdk import ( + aws_apigateway as apigateway, +) +from aws_cdk import ( + aws_dynamodb as dynamodb, +) +from aws_cdk import ( + aws_lambda as _lambda, +) +from aws_cdk import ( + aws_lambda_event_sources as lambda_event_sources, +) +from aws_cdk import ( + aws_sqs as sqs, +) +from constructs import Construct + + +class PowertoolsStack(Stack): + def __init__(self, scope: Construct, construct_id: str, environment: str = "dev", **kwargs) -> None: + super().__init__(scope, construct_id, **kwargs) + + self.env = environment + + # Shared Powertools Layer (using public layer) + self.powertools_layer = self._create_powertools_layer() + + # DynamoDB Table + self.table = self._create_dynamodb_table() + + # SQS Queue + self.queue = self._create_sqs_queue() + + # Lambda Functions + self.api_function = self._create_api_function() + self.worker_function = self._create_worker_function() + + # API Gateway + self.api = self._create_api_gateway() + + def _create_powertools_layer(self) -> _lambda.ILayerVersion: + return _lambda.LayerVersion.from_layer_version_arn( + self, + "PowertoolsLayer", + layer_version_arn="arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-x86_64:1", + ) + + def _create_dynamodb_table(self) -> dynamodb.Table: + return dynamodb.Table( + self, + "DataTable", + table_name=f"powertools-{self.env}-data", + partition_key=dynamodb.Attribute(name="pk", type=dynamodb.AttributeType.STRING), + billing_mode=dynamodb.BillingMode.PAY_PER_REQUEST, + removal_policy=RemovalPolicy.DESTROY if self.env != "prod" else RemovalPolicy.RETAIN, + ) + + def _create_sqs_queue(self) -> sqs.Queue: + return sqs.Queue( + self, + "WorkerQueue", + queue_name=f"powertools-{self.env}-worker", + visibility_timeout=Duration.seconds(180), + ) + + def _create_api_function(self) -> _lambda.Function: + function = _lambda.Function( + self, + "ApiFunction", + runtime=_lambda.Runtime.PYTHON_3_13, + handler="app.lambda_handler", + code=_lambda.Code.from_asset("src/app"), + layers=[self.powertools_layer], + timeout=Duration.seconds(30), + memory_size=512 if self.env == "prod" else 256, + environment={ + "ENVIRONMENT": self.env, + "POWERTOOLS_SERVICE_NAME": f"app-{self.env}", + "POWERTOOLS_METRICS_NAMESPACE": f"MyApp/{self.env}", + "POWERTOOLS_LOG_LEVEL": "INFO" if self.env == "prod" else "DEBUG", + "TABLE_NAME": self.table.table_name, + "QUEUE_URL": self.queue.queue_url, + }, + ) + + # Grant permissions + self.table.grant_read_write_data(function) + self.queue.grant_send_messages(function) + + return function + + def _create_worker_function(self) -> _lambda.Function: + function = _lambda.Function( + self, + "WorkerFunction", + runtime=_lambda.Runtime.PYTHON_3_13, + handler="worker.lambda_handler", + code=_lambda.Code.from_asset("src/worker"), + layers=[self.powertools_layer], + timeout=Duration.seconds(120), + memory_size=1024 if self.env == "prod" else 512, + environment={ + "ENVIRONMENT": self.env, + "POWERTOOLS_SERVICE_NAME": f"worker-{self.env}", + "POWERTOOLS_METRICS_NAMESPACE": f"MyApp/{self.env}", + "POWERTOOLS_LOG_LEVEL": "INFO" if self.env == "prod" else "DEBUG", + "TABLE_NAME": self.table.table_name, + }, + ) + + # Add SQS event source with partial failure support + function.add_event_source( + lambda_event_sources.SqsEventSource( + self.queue, + batch_size=10, + report_batch_item_failures=True, + ), + ) + + # Grant permissions + self.table.grant_read_write_data(function) + + return function + + def _create_api_gateway(self) -> apigateway.RestApi: + api = apigateway.RestApi( + self, + "ApiGateway", + rest_api_name=f"Powertools API - {self.env}", + description=f"API for {self.env} environment", + ) + + integration = apigateway.LambdaIntegration(self.api_function) + api.root.add_proxy( + default_integration=integration, + any_method=True, + ) + + return api diff --git a/examples/build_recipes/cicd/codebuild/buildspec-advanced.yml b/examples/build_recipes/cicd/codebuild/buildspec-advanced.yml new file mode 100644 index 00000000000..6441ad958f5 --- /dev/null +++ b/examples/build_recipes/cicd/codebuild/buildspec-advanced.yml @@ -0,0 +1,131 @@ +version: 0.2 + +env: + variables: + PYTHON_VERSION: "3.13" + BUILD_STAGE: "build" + parameter-store: + POWERTOOLS_VERSION: "/build/powertools-version" + +batch: + fast-fail: false + build-list: + - identifier: test + env: + variables: + BUILD_STAGE: "test" + - identifier: build_dev + env: + variables: + BUILD_STAGE: "build" + ENVIRONMENT: "dev" + depend-on: + - test + - identifier: build_prod + env: + variables: + BUILD_STAGE: "build" + ENVIRONMENT: "prod" + depend-on: + - test + +phases: + install: + runtime-versions: + python: $PYTHON_VERSION + commands: + - echo "Build stage: $BUILD_STAGE, Environment: $ENVIRONMENT" + - pip install --upgrade pip uv + + pre_build: + commands: + - | + if [ "$BUILD_STAGE" = "test" ]; then + echo "Installing test dependencies..." + uv venv test-env + source test-env/bin/activate + uv pip install aws-lambda-powertools[all]==$POWERTOOLS_VERSION pytest pytest-cov + cp -r src/ test-src/ + else + echo "Installing build dependencies..." + uv venv build-env + source build-env/bin/activate + uv pip install aws-lambda-powertools[all]==$POWERTOOLS_VERSION + uv pip install pydantic requests + fi + + build: + commands: + - | + if [ "$BUILD_STAGE" = "test" ]; then + echo "Running tests..." + source test-env/bin/activate + cd test-src + pytest tests/ --cov=. --cov-report=xml --cov-report=term + echo "Tests completed successfully" + else + echo "Building deployment package for $ENVIRONMENT..." + source build-env/bin/activate + + # Create environment-specific package + mkdir -p package-$ENVIRONMENT/ + cp -r build-env/lib/python*/site-packages/* package-$ENVIRONMENT/ + cp -r src/* package-$ENVIRONMENT/ + + # Environment-specific optimizations + if [ "$ENVIRONMENT" = "prod" ]; then + echo "Applying production optimizations..." + find package-$ENVIRONMENT/ -name "*.pyc" -delete + find package-$ENVIRONMENT/ -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true + find package-$ENVIRONMENT/ -name "tests" -type d -exec rm -rf {} + 2>/dev/null || true + find package-$ENVIRONMENT/ -name "*.dist-info" -type d -exec rm -rf {} + 2>/dev/null || true + fi + + # Create deployment ZIP + cd package-$ENVIRONMENT && zip -r ../lambda-$ENVIRONMENT.zip . && cd .. + + echo "Package size for $ENVIRONMENT: $(du -sh lambda-$ENVIRONMENT.zip)" + fi + + post_build: + commands: + - | + if [ "$BUILD_STAGE" = "build" ]; then + echo "Deploying to $ENVIRONMENT environment..." + + # Deploy to environment-specific function + aws lambda update-function-code \ + --function-name powertools-app-$ENVIRONMENT \ + --zip-file fileb://lambda-$ENVIRONMENT.zip \ + --region $AWS_DEFAULT_REGION + + # Update environment-specific configuration + LOG_LEVEL="INFO" + if [ "$ENVIRONMENT" = "dev" ]; then + LOG_LEVEL="DEBUG" + fi + + aws lambda update-function-configuration \ + --function-name powertools-app-$ENVIRONMENT \ + --environment Variables="{ + ENVIRONMENT=$ENVIRONMENT, + POWERTOOLS_SERVICE_NAME=powertools-app-$ENVIRONMENT, + POWERTOOLS_METRICS_NAMESPACE=MyApp/$ENVIRONMENT, + POWERTOOLS_LOG_LEVEL=$LOG_LEVEL + }" \ + --region $AWS_DEFAULT_REGION + + echo "Deployment to $ENVIRONMENT completed successfully!" + fi + +artifacts: + files: + - lambda-*.zip + - coverage.xml + name: lambda-artifacts-$(date +%Y-%m-%d-%H-%M-%S) + +cache: + paths: + - 'build-env/**/*' + - 'test-env/**/*' + diff --git a/examples/build_recipes/cicd/codebuild/buildspec.yml b/examples/build_recipes/cicd/codebuild/buildspec.yml new file mode 100644 index 00000000000..3605ba69805 --- /dev/null +++ b/examples/build_recipes/cicd/codebuild/buildspec.yml @@ -0,0 +1,85 @@ +version: 0.2 + +env: + variables: + PYTHON_VERSION: "3.13" + POWERTOOLS_VERSION: "3.18.0" + parameter-store: + FUNCTION_NAME: "/lambda/powertools-app/function-name" + +phases: + install: + runtime-versions: + python: $PYTHON_VERSION + commands: + - echo "Installing build dependencies..." + - pip install --upgrade pip + - pip install uv poetry # Install fast package managers + + pre_build: + commands: + - echo "Pre-build phase started on $(date)" + - echo "Python version: $(python --version)" + - echo "Installing application dependencies..." + + # Use uv for fast dependency installation + - uv venv build-env + - source build-env/bin/activate + - uv pip install aws-lambda-powertools[all]==$POWERTOOLS_VERSION + - uv pip install pydantic requests + + build: + commands: + - echo "Build started on $(date)" + - echo "Creating deployment package..." + + # Create optimized deployment package + - mkdir -p package/ + - cp -r build-env/lib/python*/site-packages/* package/ + - cp -r src/* package/ + + # Remove unnecessary files to reduce package size + - find package/ -name "*.pyc" -delete + - find package/ -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true + - find package/ -name "tests" -type d -exec rm -rf {} + 2>/dev/null || true + - find package/ -name "*.dist-info" -type d -exec rm -rf {} + 2>/dev/null || true + + # Create deployment ZIP + - cd package && zip -r ../lambda-deployment.zip . && cd .. + + # Show package info + - echo "Package size: $(du -sh lambda-deployment.zip)" + - echo "Package contents:" + - unzip -l lambda-deployment.zip | head -20 + + post_build: + commands: + - echo "Build completed on $(date)" + - echo "Deploying Lambda function..." + + # Deploy to Lambda + - aws lambda update-function-code \ + --function-name $FUNCTION_NAME \ + --zip-file fileb://lambda-deployment.zip \ + --region $AWS_DEFAULT_REGION + + # Update environment variables + - aws lambda update-function-configuration \ + --function-name $FUNCTION_NAME \ + --environment Variables="{ + POWERTOOLS_SERVICE_NAME=powertools-codebuild, + POWERTOOLS_METRICS_NAMESPACE=MyApp/CodeBuild, + POWERTOOLS_LOG_LEVEL=INFO + }" \ + --region $AWS_DEFAULT_REGION + + - echo "Deployment completed successfully!" + +artifacts: + files: + - lambda-deployment.zip + name: lambda-deployment-$(date +%Y-%m-%d-%H-%M-%S) + +cache: + paths: + - 'build-env/**/*' # Cache virtual environment for faster builds diff --git a/examples/build_recipes/cicd/github-actions/deploy-build-tools.yml b/examples/build_recipes/cicd/github-actions/deploy-build-tools.yml new file mode 100644 index 00000000000..e7ac6173955 --- /dev/null +++ b/examples/build_recipes/cicd/github-actions/deploy-build-tools.yml @@ -0,0 +1,90 @@ +name: Deploy with Different Build Tools + +on: + push: + branches: [main] + +jobs: + deploy-poetry: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Install Poetry + uses: snok/install-poetry@v1 + + - name: Build with Poetry + run: | + # Create deployment directory + mkdir -p poetry-deploy/ + + # Export and install dependencies + poetry export -f requirements.txt --output requirements.txt --without-hashes + pip install -r requirements.txt -t poetry-deploy/ + + # Copy source code + cp -r src/* poetry-deploy/ + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }} + aws-region: us-east-1 + + - name: Deploy Poetry build + uses: aws-actions/aws-lambda-deploy@v1 + with: + function-name: powertools-poetry-function + code-artifacts-dir: poetry-deploy/ + handler: app.lambda_handler + runtime: python3.13 + environment: '{"POWERTOOLS_SERVICE_NAME":"powertools-poetry","POWERTOOLS_METRICS_NAMESPACE":"MyApp","POWERTOOLS_LOG_LEVEL":"INFO"}' + role: ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} + + deploy-uv: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Build with uv (fastest) + run: | + # Install uv + pip install uv + + # Create deployment directory + mkdir -p uv-deploy/ + + # Install dependencies with uv (much faster) + uv pip install \ + --target uv-deploy/ \ + --python-version 3.13 \ + aws-lambda-powertools[all] pydantic requests + + # Copy source code + cp -r src/* uv-deploy/ + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }} + aws-region: us-east-1 + + - name: Deploy uv build + uses: aws-actions/aws-lambda-deploy@v1 + with: + function-name: powertools-uv-function + code-artifacts-dir: uv-deploy/ + handler: app.lambda_handler + runtime: python3.13 + environment: '{"POWERTOOLS_SERVICE_NAME":"powertools-uv","POWERTOOLS_METRICS_NAMESPACE":"MyApp","POWERTOOLS_LOG_LEVEL":"INFO"}' + role: ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} diff --git a/examples/build_recipes/cicd/github-actions/deploy-modern.yml b/examples/build_recipes/cicd/github-actions/deploy-modern.yml new file mode 100644 index 00000000000..648ecc91b41 --- /dev/null +++ b/examples/build_recipes/cicd/github-actions/deploy-modern.yml @@ -0,0 +1,84 @@ +name: Deploy Lambda with AWS Lambda Deploy Action + +on: + push: + branches: [main] + pull_request: + branches: [main] + +env: + AWS_REGION: us-east-1 + PYTHON_VERSION: '3.13' + +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: ${{ env.PYTHON_VERSION }} + + - name: Install dependencies + run: | + pip install aws-lambda-powertools[all] pytest pytest-cov + + - name: Run tests + run: | + pytest tests/ --cov=src/ --cov-report=xml + + - name: Upload coverage to Codecov + uses: codecov/codecov-action@v4 + with: + file: ./coverage.xml + + deploy: + needs: test + runs-on: ubuntu-latest + if: github.ref == 'refs/heads/main' + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: ${{ env.PYTHON_VERSION }} + + - name: Prepare deployment directory + run: | + # Create deployment directory with dependencies + mkdir -p deploy/ + + # Install dependencies with Linux-compatible wheels + pip install \ + --platform linux_x86_64 \ + --target deploy/ \ + --implementation cp \ + --python-version ${{ env.PYTHON_VERSION }} \ + --only-binary=:all: \ + --upgrade \ + aws-lambda-powertools[all] pydantic requests + + # Copy application code + cp -r src/* deploy/ + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }} + aws-region: ${{ env.AWS_REGION }} + + - name: Deploy to Lambda + uses: aws-actions/aws-lambda-deploy@v1 + with: + function-name: powertools-lambda-function + code-artifacts-dir: deploy/ + handler: app.lambda_handler + runtime: python3.13 + environment: '{"POWERTOOLS_SERVICE_NAME":"powertools-app","POWERTOOLS_METRICS_NAMESPACE":"MyApp","POWERTOOLS_LOG_LEVEL":"INFO"}' + timeout: 30 + memory-size: 512 + role: ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} diff --git a/examples/build_recipes/cicd/github-actions/deploy-multi-env.yml b/examples/build_recipes/cicd/github-actions/deploy-multi-env.yml new file mode 100644 index 00000000000..57ecb05cff1 --- /dev/null +++ b/examples/build_recipes/cicd/github-actions/deploy-multi-env.yml @@ -0,0 +1,79 @@ +name: Multi-Environment Lambda Deployment + +on: + push: + branches: [main, develop, staging] + +jobs: + deploy: + runs-on: ubuntu-latest + strategy: + matrix: + include: + - branch: develop + environment: dev + function-suffix: -dev + memory: 256 + timeout: 30 + log-level: DEBUG + - branch: staging + environment: staging + function-suffix: -staging + memory: 512 + timeout: 45 + log-level: INFO + - branch: main + environment: prod + function-suffix: -prod + memory: 1024 + timeout: 60 + log-level: INFO + + steps: + - uses: actions/checkout@v4 + if: github.ref == format('refs/heads/{0}', matrix.branch) + + - name: Set up Python + if: github.ref == format('refs/heads/{0}', matrix.branch) + uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Prepare deployment with uv (fast) + if: github.ref == format('refs/heads/{0}', matrix.branch) + run: | + # Install uv for fast dependency management + pip install uv + + # Create deployment directory + mkdir -p deploy-${{ matrix.environment }}/ + + # Install dependencies with uv (much faster than pip) + uv pip install \ + --target deploy-${{ matrix.environment }}/ \ + --python-version 3.13 \ + aws-lambda-powertools[all] pydantic requests + + # Copy application code + cp -r src/* deploy-${{ matrix.environment }}/ + + - name: Configure AWS credentials + if: github.ref == format('refs/heads/{0}', matrix.branch) + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }} + aws-region: us-east-1 + + - name: Deploy to Lambda + if: github.ref == format('refs/heads/{0}', matrix.branch) + uses: aws-actions/aws-lambda-deploy@v1 + with: + function-name: powertools-app${{ matrix.function-suffix }} + code-artifacts-dir: deploy-${{ matrix.environment }}/ + handler: app.lambda_handler + runtime: python3.13 + environment: '{"ENVIRONMENT":"${{ matrix.environment }}","POWERTOOLS_SERVICE_NAME":"powertools-app-${{ matrix.environment }}","POWERTOOLS_METRICS_NAMESPACE":"MyApp/${{ matrix.environment }}","POWERTOOLS_LOG_LEVEL":"${{ matrix.log-level }}"}' + timeout: ${{ matrix.timeout }} + memory-size: ${{ matrix.memory }} + role: ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} + tags: '{"Environment":"${{ matrix.environment }}","Project":"powertools-demo","ManagedBy":"github-actions"}' diff --git a/examples/build_recipes/cicd/github-actions/deploy-s3.yml b/examples/build_recipes/cicd/github-actions/deploy-s3.yml new file mode 100644 index 00000000000..cd9d296f6ab --- /dev/null +++ b/examples/build_recipes/cicd/github-actions/deploy-s3.yml @@ -0,0 +1,58 @@ +name: Deploy Lambda via S3 + +on: + push: + branches: [main] + +env: + AWS_REGION: us-east-1 + S3_BUCKET: my-lambda-deployments-bucket + +jobs: + deploy: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Prepare deployment directory + run: | + mkdir -p lambda-package/ + + # Install Powertools and dependencies + pip install \ + --platform linux_x86_64 \ + --target lambda-package/ \ + --implementation cp \ + --python-version 3.13 \ + --only-binary=:all: \ + aws-lambda-powertools[all] pydantic requests + + # Copy source code + cp -r src/* lambda-package/ + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }} + aws-region: ${{ env.AWS_REGION }} + + - name: Deploy to Lambda via S3 + uses: aws-actions/aws-lambda-deploy@v1 + with: + function-name: powertools-s3-function + code-artifacts-dir: lambda-package/ + handler: app.lambda_handler + runtime: python3.13 + s3-bucket: ${{ env.S3_BUCKET }} + s3-key: deployments/powertools-function-${{ github.sha }}.zip + environment: '{"POWERTOOLS_SERVICE_NAME":"powertools-s3","POWERTOOLS_METRICS_NAMESPACE":"MyApp","POWERTOOLS_LOG_LEVEL":"INFO"}' + timeout: 30 + memory-size: 512 + role: ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} + publish: true # Publish a new version diff --git a/examples/build_recipes/cicd/github-actions/deploy-simple.yml b/examples/build_recipes/cicd/github-actions/deploy-simple.yml new file mode 100644 index 00000000000..1f68a028ca2 --- /dev/null +++ b/examples/build_recipes/cicd/github-actions/deploy-simple.yml @@ -0,0 +1,44 @@ +name: Simple Lambda Deployment from Source + +on: + push: + branches: [main] + +env: + AWS_REGION: us-east-1 + +jobs: + deploy: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Install dependencies + run: | + # Simple pip install - the action handles the rest + pip install -r requirements.txt -t . + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }} + aws-region: ${{ env.AWS_REGION }} + + - name: Deploy to Lambda + uses: aws-actions/aws-lambda-deploy@v1 + with: + function-name: powertools-simple-function + code-artifacts-dir: . # Deploy from current directory + handler: app.lambda_handler + runtime: python3.13 + environment: '{"POWERTOOLS_SERVICE_NAME":"powertools-simple","POWERTOOLS_METRICS_NAMESPACE":"MyApp","POWERTOOLS_LOG_LEVEL":"INFO"}' + timeout: 30 + memory-size: 512 + role: ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} + dry-run: false # Set to true for validation without deployment diff --git a/examples/build_recipes/pants/basic_pants/BUILD b/examples/build_recipes/pants/basic_pants/BUILD new file mode 100644 index 00000000000..df536ae3444 --- /dev/null +++ b/examples/build_recipes/pants/basic_pants/BUILD @@ -0,0 +1,31 @@ +python_sources( + name="lambda_sources", + sources=["*.py"], +) + +python_requirement( + name="aws-lambda-powertools", + requirements=["aws-lambda-powertools[all]==3.18.0"], +) + +python_requirement( + name="pydantic", + requirements=["pydantic==2.10.4"], +) + +python_requirement( + name="requests", + requirements=["requests>=2.32.4"], +) + +pex_binary( + name="lambda_function", + entry_point="app.py:lambda_handler", + dependencies=[ + ":lambda_sources", + ":aws-lambda-powertools", + ":pydantic", + ":requests", + ], + platforms=["linux_x86_64-cp-39-cp39"], +) diff --git a/examples/build_recipes/pants/basic_pants/app_pants.py b/examples/build_recipes/pants/basic_pants/app_pants.py new file mode 100644 index 00000000000..8fb16000bd4 --- /dev/null +++ b/examples/build_recipes/pants/basic_pants/app_pants.py @@ -0,0 +1,41 @@ +from __future__ import annotations + +from typing import Any + +import requests +from pydantic import BaseModel + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.utilities.typing import LambdaContext + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + + +class TodoItem(BaseModel): + id: int + title: str + completed: bool = False + user_id: int | None = None + + +@app.get("/todos") +@tracer.capture_method +def get_todos() -> TodoItem: + """Fetch todos from external API""" + logger.info("Fetching todos from external API") + + response = requests.get("https://jsonplaceholder.typicode.com/todos") + response.raise_for_status() + + return response.json()[0] + + +@logger.inject_lambda_context +@tracer.capture_lambda_handler +@metrics.log_metrics +def lambda_handler(event: dict[str, Any], context: LambdaContext): + return app.resolve(event, context) diff --git a/examples/build_recipes/pants/basic_pants/build-pants.sh b/examples/build_recipes/pants/basic_pants/build-pants.sh new file mode 100644 index 00000000000..01f207f34ce --- /dev/null +++ b/examples/build_recipes/pants/basic_pants/build-pants.sh @@ -0,0 +1,22 @@ +#!/bin/bash + +# Build the PEX binary +pants package :lambda_function + +# The PEX file is created in dist/ +# Rename it to a more descriptive name +mv dist/lambda_function.pex lambda-pants.pex + +# For Lambda deployment, we need to extract the PEX +mkdir -p build/ +cd build/ + +# Extract PEX contents +python ../lambda-pants.pex --pex-root . --pex-path . -c "import sys; sys.exit(0)" + +# Create deployment zip +zip -r ../lambda-pants.zip . +cd .. + +echo "✅ Pants deployment package created: lambda-pants.zip" +echo "✅ Pants PEX binary created: lambda-pants.pex" diff --git a/examples/build_recipes/pants/basic_pants/pants.toml b/examples/build_recipes/pants/basic_pants/pants.toml new file mode 100644 index 00000000000..fc4b996bc8e --- /dev/null +++ b/examples/build_recipes/pants/basic_pants/pants.toml @@ -0,0 +1,17 @@ +[GLOBAL] +pants_version = "2.21.0" +backend_packages = [ + "pants.backend.python", + "pants.backend.python.lint.black", + "pants.backend.python.lint.flake8", + "pants.backend.python.typecheck.mypy", +] + +[python] +interpreter_constraints = [">=3.9,<3.14"] + +[python-infer] +use_rust_parser = true + +[source] +root_patterns = ["/"] diff --git a/examples/build_recipes/pants/multi-target/BUILD b/examples/build_recipes/pants/multi-target/BUILD new file mode 100644 index 00000000000..6f3cba9e7dc --- /dev/null +++ b/examples/build_recipes/pants/multi-target/BUILD @@ -0,0 +1,31 @@ +# Shared dependencies +python_requirement( + name="powertools", + requirements=["aws-lambda-powertools[all]==3.18.0"], +) + +# API Lambda function +python_sources( + name="api_sources", + sources=["api/*.py"], +) + +pex_binary( + name="api_lambda", + entry_point="api/handler.py:lambda_handler", + dependencies=[":api_sources", ":powertools"], + platforms=["linux_x86_64-cp-39-cp39"], +) + +# Worker Lambda function +python_sources( + name="worker_sources", + sources=["worker/*.py"], +) + +pex_binary( + name="worker_lambda", + entry_point="worker/handler.py:lambda_handler", + dependencies=[":worker_sources", ":powertools"], + platforms=["linux_x86_64-cp-39-cp39"], +) diff --git a/examples/build_recipes/pants/multi-target/app/handler.py b/examples/build_recipes/pants/multi-target/app/handler.py new file mode 100644 index 00000000000..c24d0d97cd2 --- /dev/null +++ b/examples/build_recipes/pants/multi-target/app/handler.py @@ -0,0 +1,35 @@ +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.logging import correlation_paths +from aws_lambda_powertools.metrics import MetricUnit + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + + +@app.get("/health") +def health_check(): + return {"status": "healthy", "service": "powertools-pants-api"} + + +@app.get("/metrics") +def get_metrics(): + metrics.add_metric(name="MetricsEndpointCalled", unit=MetricUnit.Count, value=1) + return {"message": "Metrics recorded"} + + +@app.post("/tasks") +def create_task(): + task_data = app.current_event.json_body + logger.info("Task created", extra={"task": task_data}) + metrics.add_metric(name="TaskCreated", unit=MetricUnit.Count, value=1) + return {"message": "Task created successfully", "task_id": task_data.get("id")} + + +@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST) +@tracer.capture_lambda_handler +@metrics.log_metrics(capture_cold_start_metric=True) +def lambda_handler(event, context): + return app.resolve(event, context) diff --git a/examples/build_recipes/pants/multi-target/build-pants-multi.sh b/examples/build_recipes/pants/multi-target/build-pants-multi.sh new file mode 100644 index 00000000000..dea42d04b10 --- /dev/null +++ b/examples/build_recipes/pants/multi-target/build-pants-multi.sh @@ -0,0 +1,22 @@ +#!/bin/bash + +# Build all Lambda functions +pants package :: + +# Process each Lambda function +for pex_file in dist/*.pex; do + base_name=$(basename "$pex_file" .pex) + + # Create build directory for this function + mkdir -p "build/$base_name" + cd "build/$base_name" + + # Extract PEX contents + python "../../$pex_file" --pex-root . --pex-path . -c "import sys; sys.exit(0)" + + # Create deployment zip + zip -r "../../$base_name.zip" . + cd ../.. + + echo "✅ Created: $base_name.zip" +done diff --git a/examples/build_recipes/pants/multi-target/worker/worker_pants.py b/examples/build_recipes/pants/multi-target/worker/worker_pants.py new file mode 100644 index 00000000000..14efa4f684e --- /dev/null +++ b/examples/build_recipes/pants/multi-target/worker/worker_pants.py @@ -0,0 +1,59 @@ +from __future__ import annotations + +import json +from typing import Any + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, process_partial_response +from aws_lambda_powertools.utilities.typing import LambdaContext + +logger = Logger() +tracer = Tracer() +metrics = Metrics() + +# Initialize batch processor for SQS +processor = BatchProcessor(event_type=EventType.SQS) + + +@tracer.capture_method +def record_handler(record): + """Process individual SQS record""" + try: + # Parse message + message_data = json.loads(record.body) + task_id = message_data.get("task_id", "unknown") + task_type = message_data.get("task_type", "default") + + logger.info("Processing task", extra={"task_id": task_id, "task_type": task_type}) + + # Simulate work based on task type + if task_type == "email": + logger.info("Sending email", extra={"task_id": task_id}) + elif task_type == "report": + logger.info("Generating report", extra={"task_id": task_id}) + else: + logger.info("Processing default task", extra={"task_id": task_id}) + + metrics.add_metric(name="TaskProcessed", unit="Count", value=1) + metrics.add_metadata(key="task_type", value=task_type) + + return {"status": "success", "task_id": task_id} + + except Exception as e: + logger.error("Task processing failed", extra={"error": str(e)}) + metrics.add_metric(name="TaskFailed", unit="Count", value=1) + raise + + +@logger.inject_lambda_context +@tracer.capture_lambda_handler +@metrics.log_metrics +def lambda_handler(event: dict[str, Any], context: LambdaContext): + """Process SQS messages using BatchProcessor""" + + return process_partial_response( + event=event, + record_handler=record_handler, + processor=processor, + context=context, + ) diff --git a/examples/build_recipes/pip/app_pip.py b/examples/build_recipes/pip/app_pip.py new file mode 100644 index 00000000000..386bf4f0f83 --- /dev/null +++ b/examples/build_recipes/pip/app_pip.py @@ -0,0 +1,28 @@ +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.logging import correlation_paths +from aws_lambda_powertools.metrics import MetricUnit + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + + +@app.get("/hello") +def hello(): + logger.info("Hello World API called") + metrics.add_metric(name="HelloWorldInvocations", unit=MetricUnit.Count, value=1) + return {"message": "Hello World from Powertools!"} + + +@app.get("/health") +def health_check(): + return {"status": "healthy", "service": "powertools-pip-example"} + + +@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST) +@tracer.capture_lambda_handler +@metrics.log_metrics(capture_cold_start_metric=True) +def lambda_handler(event, context): + return app.resolve(event, context) diff --git a/examples/build_recipes/pip/build-cross-platform.sh b/examples/build_recipes/pip/build-cross-platform.sh new file mode 100644 index 00000000000..a5b6990e122 --- /dev/null +++ b/examples/build_recipes/pip/build-cross-platform.sh @@ -0,0 +1,24 @@ +#!/bin/bash + +# Build for Lambda x86_64 (most common) +mkdir -p build-x86_64/ +pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build-x86_64/ \ + -r requirements.txt + +# Build for Lambda ARM64 (Graviton2) +mkdir -p build-arm64/ +pip install --platform manylinux2014_aarch64 --only-binary=:all: \ + --python-version 3.13 --target build-arm64/ \ + -r requirements.txt + +# Copy application code to both builds +cp app_pip.py build-x86_64/ +cp app_pip.py build-arm64/ + +# Create deployment packages +cd build-x86_64 && zip -r ../lambda-x86_64.zip . && cd .. +cd build-arm64 && zip -r ../lambda-arm64.zip . && cd .. + +echo "✅ x86_64 package: lambda-x86_64.zip" +echo "✅ ARM64 package: lambda-arm64.zip" diff --git a/examples/build_recipes/pip/build-with-layer.sh b/examples/build_recipes/pip/build-with-layer.sh new file mode 100644 index 00000000000..99c3c69114b --- /dev/null +++ b/examples/build_recipes/pip/build-with-layer.sh @@ -0,0 +1,19 @@ +#!/bin/bash + +# Build Lambda Layer with compatible wheels +mkdir -p layer/python/ +pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target layer/python/ \ + -r requirements-layer.txt +cd layer && zip -r ../powertools-layer.zip . && cd .. + +# Build application package (smaller without Powertools) +mkdir -p build/ +pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build/ \ + -r requirements-app.txt +cp app_pip.py build/ +cd build && zip -r ../lambda-app.zip . && cd .. + +echo "✅ Layer created: powertools-layer.zip" +echo "✅ App package created: lambda-app.zip" diff --git a/examples/build_recipes/pip/build.sh b/examples/build_recipes/pip/build.sh new file mode 100755 index 00000000000..c93d62d5e26 --- /dev/null +++ b/examples/build_recipes/pip/build.sh @@ -0,0 +1,17 @@ +#!/bin/bash + +# Create build directory +mkdir -p build/ + +# Install dependencies with Lambda-compatible wheels +pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build/ \ + -r requirements.txt + +# Copy application code +cp app_pip.py build/ + +# Create deployment package +cd build && zip -r ../lambda-deployment.zip . && cd .. + +echo "✅ Deployment package created: lambda-deployment.zip" diff --git a/examples/build_recipes/pip/requirements-app.txt b/examples/build_recipes/pip/requirements-app.txt new file mode 100644 index 00000000000..85b38c33203 --- /dev/null +++ b/examples/build_recipes/pip/requirements-app.txt @@ -0,0 +1,2 @@ +pydantic==2.10.4 +requests>=2.32.4 diff --git a/examples/build_recipes/pip/requirements-layer.txt b/examples/build_recipes/pip/requirements-layer.txt new file mode 100644 index 00000000000..e2b89392f37 --- /dev/null +++ b/examples/build_recipes/pip/requirements-layer.txt @@ -0,0 +1 @@ +aws-lambda-powertools[all]==3.18.0 diff --git a/examples/build_recipes/pip/requirements.txt b/examples/build_recipes/pip/requirements.txt new file mode 100644 index 00000000000..3b2777aa329 --- /dev/null +++ b/examples/build_recipes/pip/requirements.txt @@ -0,0 +1,3 @@ +aws-lambda-powertools[all]==3.18.0 +pydantic==2.10.4 +requests>=2.32.4 diff --git a/examples/build_recipes/poetry/Dockerfile.poetry b/examples/build_recipes/poetry/Dockerfile.poetry new file mode 100644 index 00000000000..0a40f8fa11f --- /dev/null +++ b/examples/build_recipes/poetry/Dockerfile.poetry @@ -0,0 +1,18 @@ +#Public Lambda image +FROM public.ecr.aws/lambda/python@sha256:7e7f098baa11a527fbe59f33f4ed032a36b6e87b22ea73da1175522095885f74 + +# Set workdir file +WORKDIR /tmp/app + +# Copy poetry files +COPY pyproject.toml poetry.lock ./ + +# Configure poetry and install dependencies +RUN poetry config virtualenvs.create false \ + pip install poetry \ + poetry install --only=main --no-root + +# Copy application code +COPY app_poetry.py ./ + +CMD ["app_poetry.lambda_handler"] diff --git a/examples/build_recipes/poetry/app_poetry.py b/examples/build_recipes/poetry/app_poetry.py new file mode 100644 index 00000000000..d570cca39cf --- /dev/null +++ b/examples/build_recipes/poetry/app_poetry.py @@ -0,0 +1,40 @@ +from typing import Optional + +from pydantic import BaseModel + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.logging import correlation_paths +from aws_lambda_powertools.metrics import MetricUnit + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + + +class UserModel(BaseModel): + name: str + email: str + age: Optional[int] = None + + +@app.post("/users") +def create_user(user: UserModel): + logger.info("Creating user", extra={"user": user.model_dump()}) + metrics.add_metric(name="UserCreated", unit=MetricUnit.Count, value=1) + return {"message": f"User {user.name} created successfully", "user": user.model_dump()} + + +@app.get("/users") +def list_users(): + logger.info("Listing users") + metrics.add_metric(name="UsersListed", unit=MetricUnit.Count, value=1) + return {"users": [{"name": "John Doe", "email": "john@example.com", "age": 30}]} + + +@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST) +@tracer.capture_lambda_handler +@metrics.log_metrics(capture_cold_start_metric=True) +def lambda_handler(event, context): + return app.resolve(event, context) diff --git a/examples/build_recipes/poetry/build-poetry-cross-platform.sh b/examples/build_recipes/poetry/build-poetry-cross-platform.sh new file mode 100644 index 00000000000..d63850b90e1 --- /dev/null +++ b/examples/build_recipes/poetry/build-poetry-cross-platform.sh @@ -0,0 +1,30 @@ +#!/bin/bash + +# Export requirements for Lambda +poetry export -f requirements.txt --output requirements.txt --without-hashes + +# Build for Lambda x86_64 (most common) +mkdir -p build-x86_64/ +pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build-x86_64/ \ + -r requirements.txt + +# Build for Lambda ARM64 (Graviton2) +mkdir -p build-arm64/ +pip install --platform manylinux2014_aarch64 --only-binary=:all: \ + --python-version 3.13 --target build-arm64/ \ + -r requirements.txt + +# Copy application code to both builds +cp app_poetry.py build-x86_64/ +cp app_poetry.py build-arm64/ + +# Create deployment packages +cd build-x86_64 && zip -r ../lambda-poetry-x86_64.zip . && cd .. +cd build-arm64 && zip -r ../lambda-poetry-arm64.zip . && cd .. + +# Cleanup +rm requirements.txt + +echo "✅ x86_64 package: lambda-poetry-x86_64.zip" +echo "✅ ARM64 package: lambda-poetry-arm64.zip" diff --git a/examples/build_recipes/poetry/build-poetry-native.sh b/examples/build_recipes/poetry/build-poetry-native.sh new file mode 100644 index 00000000000..0534c9d4747 --- /dev/null +++ b/examples/build_recipes/poetry/build-poetry-native.sh @@ -0,0 +1,21 @@ +#!/bin/bash + +# Create build directory +mkdir -p build/ + +# Install dependencies directly to build directory using Poetry +# Note: This method may not handle cross-platform compatibility as well +poetry install --only=main --no-root + +# Copy installed packages from virtual environment +VENV_PATH=$(poetry env info --path) +cp -r "$VENV_PATH/lib/python*/site-packages"/* build/ + +# Copy application code +cp app_poetry.py build/ + +# Create deployment package +cd build && zip -r ../lambda-poetry-native.zip . && cd .. + +echo "✅ Poetry native deployment package created: lambda-poetry-native.zip" +echo "⚠️ Warning: This method may have cross-platform compatibility issues" diff --git a/examples/build_recipes/poetry/build-with-poetry-docker.sh b/examples/build_recipes/poetry/build-with-poetry-docker.sh new file mode 100644 index 00000000000..eaadded1ed0 --- /dev/null +++ b/examples/build_recipes/poetry/build-with-poetry-docker.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +# Build Docker image +docker build -t lambda-powertools-app -f Dockerfile.poetry . + +# Create container and extract files +docker create --name temp-container lambda-powertools-app +docker cp temp-container:/var/task ./build +docker rm temp-container + +# Create deployment package +cd build && zip -r ../lambda-docker.zip . && cd .. + +echo "✅ Docker-based deployment package created: lambda-docker.zip" diff --git a/examples/build_recipes/poetry/build-with-poetry.sh b/examples/build_recipes/poetry/build-with-poetry.sh new file mode 100644 index 00000000000..8826f3318a4 --- /dev/null +++ b/examples/build_recipes/poetry/build-with-poetry.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +# Export requirements for Lambda +poetry export -f requirements.txt --output requirements.txt --without-hashes + +# Create build directory +mkdir -p build/ + +# Install dependencies with Lambda-compatible wheels +pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build/ \ + -r requirements.txt + +# Copy application code +cp app_poetry.py build/ + +# Create deployment package +cd build && zip -r ../lambda-poetry.zip . && cd .. + +# Cleanup +rm requirements.txt + +echo "✅ Poetry deployment package created: lambda-poetry.zip" diff --git a/examples/build_recipes/poetry/pyproject.toml b/examples/build_recipes/poetry/pyproject.toml new file mode 100644 index 00000000000..2fa598465db --- /dev/null +++ b/examples/build_recipes/poetry/pyproject.toml @@ -0,0 +1,23 @@ +[tool.poetry] +name = "lambda-powertools-app" +version = "0.1.0" +description = "Lambda function with Powertools" + +[tool.poetry.dependencies] +python = "^3.10" +aws-lambda-powertools = {extras = ["all"], version = "^3.18.0"} +pydantic = "^2.10.0" +requests = "^2.32.0" + +[tool.poetry.group.dev.dependencies] +pytest = "^8.0.0" +black = "^24.0.0" +mypy = "^1.8.0" + +[tool.poetry.requires-plugins] +poetry-plugin-export = ">=1.8" + +[build-system] +requires = ["poetry-core"] +build-backend = "poetry.core.masonry.api" + diff --git a/examples/build_recipes/sam/multi-env/template.yaml b/examples/build_recipes/sam/multi-env/template.yaml new file mode 100644 index 00000000000..31b8d6aa588 --- /dev/null +++ b/examples/build_recipes/sam/multi-env/template.yaml @@ -0,0 +1,91 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 + +Parameters: + Environment: + Type: String + Default: dev + AllowedValues: [dev, staging, prod] + Description: Environment name + + LogLevel: + Type: String + Default: INFO + AllowedValues: [DEBUG, INFO, WARNING, ERROR] + Description: Log level for Lambda functions + +Mappings: + EnvironmentMap: + dev: + MemorySize: 256 + Timeout: 30 + staging: + MemorySize: 512 + Timeout: 60 + prod: + MemorySize: 1024 + Timeout: 120 + +Globals: + Function: + Runtime: python3.13 + MemorySize: !FindInMap [EnvironmentMap, !Ref Environment, MemorySize] + Timeout: !FindInMap [EnvironmentMap, !Ref Environment, Timeout] + Environment: + Variables: + ENVIRONMENT: !Ref Environment + POWERTOOLS_SERVICE_NAME: !Sub "${AWS::StackName}-${Environment}" + POWERTOOLS_METRICS_NAMESPACE: !Sub "MyApp/${Environment}" + POWERTOOLS_LOG_LEVEL: !Ref LogLevel + POWERTOOLS_DEV: !If [IsDev, "true", "false"] + +Conditions: + IsDev: !Equals [!Ref Environment, "dev"] + IsProd: !Equals [!Ref Environment, "prod"] + +Resources: + # Dependencies Layer for application dependencies + DependenciesLayer: + Type: AWS::Serverless::LayerVersion + Properties: + LayerName: !Sub "${AWS::StackName}-${Environment}-dependencies" + Description: !Sub "Application dependencies for ${Environment}" + ContentUri: layers/dependencies/ + CompatibleRuntimes: + - python3.13 + RetentionPolicy: Delete + + ApiFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: src/ + Handler: app.lambda_handler + Layers: + - arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-x86_64:1 + - !Ref DependenciesLayer + Events: + ApiEvent: + Type: Api + Properties: + Path: /{proxy+} + Method: ANY + Environment: + Variables: + TABLE_NAME: !Ref DynamoTable + + DynamoTable: + Type: AWS::DynamoDB::Table + Properties: + TableName: !Sub "${AWS::StackName}-${Environment}-data" + BillingMode: !If [IsProd, "PROVISIONED", "PAY_PER_REQUEST"] + AttributeDefinitions: + - AttributeName: pk + AttributeType: S + KeySchema: + - AttributeName: pk + KeyType: HASH + ProvisionedThroughput: !If + - IsProd + - ReadCapacityUnits: 5 + WriteCapacityUnits: 5 + - !Ref AWS::NoValue diff --git a/examples/build_recipes/sam/no-layers/build-sam-no-layers.sh b/examples/build_recipes/sam/no-layers/build-sam-no-layers.sh new file mode 100644 index 00000000000..53992bf3e71 --- /dev/null +++ b/examples/build_recipes/sam/no-layers/build-sam-no-layers.sh @@ -0,0 +1,9 @@ +#!/bin/bash + +echo "🏗️ Building SAM application without layers..." + +# Build and deploy (SAM will handle dependency installation) +sam build --use-container +sam deploy --guided + +echo "✅ SAM application deployed successfully (no layers)" diff --git a/examples/build_recipes/sam/no-layers/requirements.txt b/examples/build_recipes/sam/no-layers/requirements.txt new file mode 100644 index 00000000000..3b2777aa329 --- /dev/null +++ b/examples/build_recipes/sam/no-layers/requirements.txt @@ -0,0 +1,3 @@ +aws-lambda-powertools[all]==3.18.0 +pydantic==2.10.4 +requests>=2.32.4 diff --git a/examples/build_recipes/sam/no-layers/src/app_sam_no_layer.py b/examples/build_recipes/sam/no-layers/src/app_sam_no_layer.py new file mode 100644 index 00000000000..dcda4cede13 --- /dev/null +++ b/examples/build_recipes/sam/no-layers/src/app_sam_no_layer.py @@ -0,0 +1,38 @@ +from typing import Optional + +from pydantic import BaseModel + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.logging import correlation_paths +from aws_lambda_powertools.metrics import MetricUnit + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + + +class UserModel(BaseModel): + name: str + email: str + age: Optional[int] = None + + +@app.get("/health") +def health_check(): + return {"status": "healthy", "service": "powertools-sam"} + + +@app.post("/users") +def create_user(user: UserModel): + logger.info("Creating user", extra={"user": user.model_dump()}) + metrics.add_metric(name="UserCreated", unit=MetricUnit.Count, value=1) + return {"message": f"User {user.name} created successfully"} + + +@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST) +@tracer.capture_lambda_handler +@metrics.log_metrics(capture_cold_start_metric=True) +def lambda_handler(event, context): + return app.resolve(event, context) diff --git a/examples/build_recipes/sam/no-layers/template.yaml b/examples/build_recipes/sam/no-layers/template.yaml new file mode 100644 index 00000000000..02eccd08c7f --- /dev/null +++ b/examples/build_recipes/sam/no-layers/template.yaml @@ -0,0 +1,35 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 + +Globals: + Function: + Runtime: python3.13 + Timeout: 30 + MemorySize: 512 + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: !Ref AWS::StackName + POWERTOOLS_METRICS_NAMESPACE: MyApp + POWERTOOLS_LOG_LEVEL: INFO + +Resources: + # Single Lambda Function with all dependencies included + ApiFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: src/ + Handler: app_sam_no_layer.lambda_handler + Events: + ApiEvent: + Type: Api + Properties: + Path: /{proxy+} + Method: ANY + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: api-service + +Outputs: + ApiUrl: + Description: API Gateway endpoint URL + Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/" diff --git a/examples/build_recipes/sam/with-layers/build-sam-with-layers.sh b/examples/build_recipes/sam/with-layers/build-sam-with-layers.sh new file mode 100644 index 00000000000..c6d47af6809 --- /dev/null +++ b/examples/build_recipes/sam/with-layers/build-sam-with-layers.sh @@ -0,0 +1,27 @@ +#!/bin/bash + +echo "🏗️ Building SAM application with layers..." + +# Build Dependencies layer (Powertools uses public layer ARN) +echo "Building Dependencies layer..." +mkdir -p layers/dependencies/python +pip install pydantic requests -t layers/dependencies/python/ + +# Optimize layers (remove unnecessary files) +echo "Optimizing layers..." +find layers/ -name "*.pyc" -delete +find layers/ -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true +find layers/ -name "tests" -type d -exec rm -rf {} + 2>/dev/null || true +find layers/ -name "*.dist-info" -type d -exec rm -rf {} + 2>/dev/null || true + +# Build and deploy +sam build --use-container +sam deploy --guided + +echo "✅ SAM application with layers deployed successfully" + +# Show layer sizes +echo "" +echo "📊 Layer sizes:" +echo "Powertools: Using public layer ARN (no local build needed)" +du -sh layers/dependencies/ diff --git a/examples/build_recipes/sam/with-layers/layers/dependencies/requirements.txt b/examples/build_recipes/sam/with-layers/layers/dependencies/requirements.txt new file mode 100644 index 00000000000..85b38c33203 --- /dev/null +++ b/examples/build_recipes/sam/with-layers/layers/dependencies/requirements.txt @@ -0,0 +1,2 @@ +pydantic==2.10.4 +requests>=2.32.4 diff --git a/examples/build_recipes/sam/with-layers/samconfig.toml b/examples/build_recipes/sam/with-layers/samconfig.toml new file mode 100644 index 00000000000..4276a9a962b --- /dev/null +++ b/examples/build_recipes/sam/with-layers/samconfig.toml @@ -0,0 +1,26 @@ +version = 0.1 + +[default.global.parameters] +stack_name = "powertools-lambda-app" + +[default.build.parameters] +cached = true +parallel = true + +[default.deploy.parameters] +capabilities = "CAPABILITY_IAM" +confirm_changeset = true +resolve_s3 = true +region = "us-east-1" + +[default.package.parameters] +resolve_s3 = true + +[default.sync.parameters] +watch = true + +[default.local_start_api.parameters] +warm_containers = "EAGER" + +[default.local_start_lambda.parameters] +warm_containers = "EAGER" diff --git a/examples/build_recipes/sam/with-layers/src/app/app_sam_layer.py b/examples/build_recipes/sam/with-layers/src/app/app_sam_layer.py new file mode 100644 index 00000000000..9d4b39e63a1 --- /dev/null +++ b/examples/build_recipes/sam/with-layers/src/app/app_sam_layer.py @@ -0,0 +1,50 @@ +from typing import Optional + +import requests +from pydantic import BaseModel + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.logging import correlation_paths +from aws_lambda_powertools.metrics import MetricUnit + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + + +class UserModel(BaseModel): + name: str + email: str + age: Optional[int] = None + + +@app.get("/health") +def health_check(): + return {"status": "healthy", "service": "powertools-sam-layers"} + + +@app.post("/users") +def create_user(user: UserModel): + logger.info("Creating user", extra={"user": user.model_dump()}) + metrics.add_metric(name="UserCreated", unit=MetricUnit.Count, value=1) + return {"message": f"User {user.name} created successfully"} + + +@app.get("/external") +@tracer.capture_method +def fetch_external_data(): + """Example using requests from dependencies layer""" + response = requests.get("https://httpbin.org/json") + data = response.json() + + metrics.add_metric(name="ExternalApiCalled", unit=MetricUnit.Count, value=1) + return {"external_data": data} + + +@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST) +@tracer.capture_lambda_handler +@metrics.log_metrics(capture_cold_start_metric=True) +def lambda_handler(event, context): + return app.resolve(event, context) diff --git a/examples/build_recipes/sam/with-layers/src/worker/worker_sam_layer.py b/examples/build_recipes/sam/with-layers/src/worker/worker_sam_layer.py new file mode 100644 index 00000000000..d5abbd309ac --- /dev/null +++ b/examples/build_recipes/sam/with-layers/src/worker/worker_sam_layer.py @@ -0,0 +1,72 @@ +from __future__ import annotations + +import json +from typing import Any + +from pydantic import BaseModel, ValidationError + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, process_partial_response +from aws_lambda_powertools.utilities.typing import LambdaContext + +logger = Logger() +tracer = Tracer() +metrics = Metrics() + +# Initialize batch processor for SQS +processor = BatchProcessor(event_type=EventType.SQS) + + +class WorkerMessage(BaseModel): + task_id: str + task_type: str + payload: dict + + +@tracer.capture_method +def record_handler(record): + """Process individual SQS record""" + try: + # Parse and validate message + message_data = json.loads(record.body) + worker_message = WorkerMessage(**message_data) + + logger.info("Processing task", extra={"task_id": worker_message.task_id, "task_type": worker_message.task_type}) + + # Simulate work based on task type + if worker_message.task_type == "email": + # Process email task + logger.info("Sending email", extra={"task_id": worker_message.task_id}) + elif worker_message.task_type == "report": + # Process report task + logger.info("Generating report", extra={"task_id": worker_message.task_id}) + else: + logger.warning("Unknown task type", extra={"task_type": worker_message.task_type}) + + metrics.add_metric(name="TaskProcessed", unit="Count", value=1) + metrics.add_metadata(key="task_type", value=worker_message.task_type) + + return {"status": "success", "task_id": worker_message.task_id} + + except ValidationError as e: + logger.error("Invalid message format", extra={"error": str(e)}) + metrics.add_metric(name="TaskFailed", unit="Count", value=1) + raise + except Exception as e: + logger.error("Task processing failed", extra={"error": str(e)}) + metrics.add_metric(name="TaskFailed", unit="Count", value=1) + raise + + +@logger.inject_lambda_context +@tracer.capture_lambda_handler +@metrics.log_metrics +def lambda_handler(event: dict[str, Any], context: LambdaContext): + """Process SQS messages using BatchProcessor""" + + return process_partial_response( + event=event, + record_handler=record_handler, + processor=processor, + context=context, + ) diff --git a/examples/build_recipes/sam/with-layers/template.yaml b/examples/build_recipes/sam/with-layers/template.yaml new file mode 100644 index 00000000000..21a8aae539a --- /dev/null +++ b/examples/build_recipes/sam/with-layers/template.yaml @@ -0,0 +1,81 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 + +Globals: + Function: + Runtime: python3.13 + Timeout: 30 + MemorySize: 512 + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: !Ref AWS::StackName + POWERTOOLS_METRICS_NAMESPACE: MyApp + POWERTOOLS_LOG_LEVEL: INFO + +Resources: + # Dependencies Layer (pydantic, requests, etc.) + DependenciesLayer: + Type: AWS::Serverless::LayerVersion + Properties: + LayerName: !Sub "${AWS::StackName}-dependencies" + Description: Application dependencies + ContentUri: layers/dependencies/ + CompatibleRuntimes: + - python3.13 + RetentionPolicy: Delete + + # API Lambda Function (lightweight - only app code) + ApiFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: src/app/ + Handler: app_sam_layer.lambda_handler + Layers: + - arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-x86_64:21 + - !Ref DependenciesLayer + Events: + ApiEvent: + Type: Api + Properties: + Path: /{proxy+} + Method: ANY + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: api-service + + # Background Worker Function + WorkerFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: src/worker/ + Handler: worker_sam_layer.lambda_handler + Layers: + - arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-x86_64:21 + - !Ref DependenciesLayer + Events: + SQSEvent: + Type: SQS + Properties: + Queue: !GetAtt WorkerQueue.Arn + BatchSize: 10 + FunctionResponseTypes: + - ReportBatchItemFailures + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: worker-service + + # SQS Queue for worker + WorkerQueue: + Type: AWS::SQS::Queue + Properties: + QueueName: !Sub "${AWS::StackName}-worker-queue" + VisibilityTimeout: 180 + +Outputs: + ApiUrl: + Description: API Gateway endpoint URL + Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/" + + WorkerQueueUrl: + Description: SQS Queue URL for worker + Value: !Ref WorkerQueue diff --git a/examples/build_recipes/uv/app_uv.py b/examples/build_recipes/uv/app_uv.py new file mode 100644 index 00000000000..de382edf681 --- /dev/null +++ b/examples/build_recipes/uv/app_uv.py @@ -0,0 +1,30 @@ +from __future__ import annotations + +from typing import Any + +from aws_lambda_powertools import Logger, Metrics, Tracer +from aws_lambda_powertools.event_handler import APIGatewayRestResolver +from aws_lambda_powertools.utilities.typing import LambdaContext + +logger = Logger() +tracer = Tracer() +metrics = Metrics() +app = APIGatewayRestResolver() + + +@app.get("/health") +def health_check(): + return {"status": "healthy", "service": "lambda-powertools-uv"} + + +@app.get("/metrics") +def get_metrics(): + metrics.add_metric(name="MetricsEndpointCalled", unit="Count", value=1) + return {"message": "Metrics recorded"} + + +@logger.inject_lambda_context +@tracer.capture_lambda_handler +@metrics.log_metrics +def lambda_handler(event: dict[str, Any], context: LambdaContext): + return app.resolve(event, context) diff --git a/examples/build_recipes/uv/build-uv-cross-platform.sh b/examples/build_recipes/uv/build-uv-cross-platform.sh new file mode 100644 index 00000000000..838e34c0665 --- /dev/null +++ b/examples/build_recipes/uv/build-uv-cross-platform.sh @@ -0,0 +1,24 @@ +#!/bin/bash + +# Build for Lambda x86_64 (most common) +mkdir -p build-x86_64/ +uv pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build-x86_64/ \ + -e . + +# Build for Lambda ARM64 (Graviton2) +mkdir -p build-arm64/ +uv pip install --platform manylinux2014_aarch64 --only-binary=:all: \ + --python-version 3.13 --target build-arm64/ \ + -e . + +# Copy application code to both builds +cp app_uv.py build-x86_64/ +cp app_uv.py build-arm64/ + +# Create deployment packages +cd build-x86_64 && zip -r ../lambda-uv-x86_64.zip . && cd .. +cd build-arm64 && zip -r ../lambda-uv-arm64.zip . && cd .. + +echo "✅ x86_64 package: lambda-uv-x86_64.zip" +echo "✅ ARM64 package: lambda-uv-arm64.zip" diff --git a/examples/build_recipes/uv/build-uv-locked.sh b/examples/build_recipes/uv/build-uv-locked.sh new file mode 100644 index 00000000000..dcaa3a90902 --- /dev/null +++ b/examples/build_recipes/uv/build-uv-locked.sh @@ -0,0 +1,26 @@ +#!/bin/bash + +# Generate lock file for reproducible builds +uv lock + +# Export to requirements.txt for Lambda +uv export --format requirements-txt --no-hashes > requirements.txt + +# Create build directory +mkdir -p build/ + +# Install to build directory with Lambda-compatible wheels +uv pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build/ \ + -r requirements.txt + +# Copy application code +cp app_uv.py build/ + +# Create deployment package +cd build && zip -r ../lambda-uv-locked.zip . && cd .. + +# Cleanup +rm requirements.txt + +echo "✅ uv locked deployment package created: lambda-uv-locked.zip" diff --git a/examples/build_recipes/uv/build-uv.sh b/examples/build_recipes/uv/build-uv.sh new file mode 100644 index 00000000000..1ffa413f547 --- /dev/null +++ b/examples/build_recipes/uv/build-uv.sh @@ -0,0 +1,17 @@ +#!/bin/bash + +# Create build directory +mkdir -p build/ + +# Install dependencies with Lambda-compatible wheels +uv pip install --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.13 --target build/ \ + -e . + +# Copy application code +cp app_uv.py build/ + +# Create deployment package +cd build && zip -r ../lambda-uv.zip . && cd .. + +echo "✅ uv deployment package created: lambda-uv.zip" diff --git a/examples/build_recipes/uv/pyproject.toml b/examples/build_recipes/uv/pyproject.toml new file mode 100644 index 00000000000..ff97a0ac5fc --- /dev/null +++ b/examples/build_recipes/uv/pyproject.toml @@ -0,0 +1,17 @@ +[project] +name = "lambda-powertools-uv" +version = "0.1.0" +description = "Lambda function with Powertools using uv" +requires-python = ">=3.9" +dependencies = [ + "aws-lambda-powertools[all]>=3.18.0", + "pydantic>=2.10.0", + "requests>=2.32.0", +] + +[project.optional-dependencies] +dev = [ + "pytest>=8.0.0", + "black>=24.0.0", + "mypy>=1.8.0", +] diff --git a/mkdocs.yml b/mkdocs.yml index ce2f2aac5d6..9551a7507b3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -9,11 +9,22 @@ nav: - Homepage: - index.md - Changelog: changelog.md - - API reference: api/" target="_blank - - Upgrade guide: upgrade.md - - We Made This (Community): we_made_this.md + - Tutorial: tutorial/index.md - Workshop 🆕: https://s12d.com/powertools-for-aws-lambda-workshop" target="_blank - - Roadmap: roadmap.md + - Build recipes: + - build_recipes/index.md + - Getting started: build_recipes/getting-started.md + - Cross-platform builds: build_recipes/cross-platform.md + - Build with pip: build_recipes/build-with-pip.md + - Build with poetry: build_recipes/build-with-poetry.md + - Build with uv: build_recipes/build-with-uv.md + - Build with SAM: build_recipes/build-with-sam.md + - Build with CDK: build_recipes/build-with-cdk.md + - Build with Pants: build_recipes/build-with-pants.md + - Performance optimization: build_recipes/performance-optimization.md + - CI/CD integration: build_recipes/cicd-integration.md + - Troubleshooting: build_recipes/troubleshooting.md + - API reference: api/" target="_blank - Features: - core/tracer.md - core/logger.md @@ -40,10 +51,13 @@ nav: - utilities/middleware_factory.md - utilities/jmespath_functions.md - CloudFormation Custom Resources: https://github.com/aws-cloudformation/custom-resource-helper" target="_blank + - Upgrade guide: upgrade.md + - We Made This (Community): we_made_this.md + - Roadmap: roadmap.md - Resources: - "llms.txt": ./llms.txt - "llms.txt (full version)": ./llms-full.txt - - Tutorial: tutorial/index.md + - Processes: - Security: security.md - Automation: automation.md @@ -233,6 +247,19 @@ plugins: - utilities/jmespath_functions.md Tutorial: - tutorial/index.md + Build recipes: + - build_recipes/index.md + - build_recipes/getting-started.md + - build_recipes/cross-platform.md + - build_recipes/build-with-pip.md + - build_recipes/build-with-poetry.md + - build_recipes/build-with-uv.md + - build_recipes/build-with-sam.md + - build_recipes/build-with-cdk.md + - build_recipes/build-with-pants.md + - build_recipes/performance-optimization.md + - build_recipes/cicd-integration.md + - build_recipes/troubleshooting.md - mkdocstrings: default_handler: python