Add dynamic resource generation from AWS CloudFormation schemas

[Jordi Soucheiron (jsoucheiron)]

Summary

• Adds a schema_generator module that dynamically creates pycfmodel Resource classes from AWS CloudFormation resource provider schemas
• Downloads and caches schemas from AWS (https://schema.cloudformation.{region}.amazonaws.com/CloudformationSchema.zip)
• Generates Pydantic models with proper type hints from JSON schemas
• Filters out read-only properties automatically
• Adds strict validation for CloudFormation intrinsic functions to detect malformed templates
• Adds automatic dynamic resource generation during template parsing for unmodeled resource types

Features Supported

Dynamic Resource Generation During Parsing

When enabled, pycfmodel will automatically generate typed models for resource types that are not explicitly modeled:

from pycfmodel import parse, enable_dynamic_generation

# Enable dynamic generation (disabled by default)
enable_dynamic_generation()

template = {
    "Resources": {
        "MyFunction": {
            "Type": "AWS::Lambda::Function",  # Not explicitly modeled
            "Properties": {
                "FunctionName": "my-function",
                "Runtime": "python3.12",
                "Role": "arn:aws:iam::123456789012:role/lambda-role",
                "Code": {"ZipFile": "def handler(event, context): pass"},
            },
        }
    }
}

model = parse(template)
# MyFunction is now a typed Resource, not GenericResource!
assert model.Resources["MyFunction"].Properties.FunctionName == "my-function"

Available functions:

• enable_dynamic_generation() - Enable dynamic generation
• disable_dynamic_generation() - Disable dynamic generation (default)
• is_dynamic_generation_enabled() - Check current state
• clear_dynamic_model_cache() - Clear cached dynamically generated models

Intrinsic Function Validation

All CloudFormation intrinsic functions are now validated at parse time:

Function	Validation
Ref	Must be a non-empty string
Fn::Sub	String or [string, dict] format
Fn::GetAtt	[resource, attr] list or "resource.attr" string
Fn::Join	[delimiter, list] with string delimiter
Fn::Select	[index, list] format
Fn::Split	[delimiter, string] with string delimiter
Fn::If	[condition, true_value, false_value] - exactly 3 elements
Fn::And / Fn::Or	List of 2-10 conditions
Fn::Not	List with exactly 1 condition
Fn::Equals	List with exactly 2 values
Fn::FindInMap	[map, key1, key2] - exactly 3 elements
Fn::Base64	String or function
Fn::GetAZs	String or function
Fn::ImportValue	String or function
Condition	Non-empty string

Manual Dynamic Resource Generation

• Resource attributes: DependsOn, Condition, DeletionPolicy, UpdateReplacePolicy, Metadata
• Optional properties, arrays, and nested objects

from pycfmodel.schema_generator import generate_resource_from_schema

DynamicLambdaFunction = generate_resource_from_schema("AWS::Lambda::Function")
function = DynamicLambdaFunction(
    Type="AWS::Lambda::Function",
    Properties={"FunctionName": "my-function", "Runtime": "python3.12", ...}
)

Malformed Template Detection

Invalid intrinsic functions are now rejected at parse time:

# This will raise a ValidationError
bucket = S3Bucket(
    Type="AWS::S3::Bucket",
    Properties={"BucketName": {"Ref": ["invalid", "list"]}}  # Ref must be a string
)
# ValidationError: Ref value must be a string, got list

Test plan

• Unit tests for schema retrieval
• Tests for all CloudFormation intrinsic functions validation (71 tests)
• Tests for CloudFormation conditions
• Tests for resource metadata attributes
• Integration tests for multiple resource types (S3, Lambda, DynamoDB, SNS, SQS, EC2)
• Full ECS+ALB CloudFormation template test with 11 resource types
• Dynamic resource generation during parsing tests (16 tests)
  • Toggle enable/disable functionality
  • Unmodeled resources become typed resources when enabled
  • Explicitly modeled resources still use explicit models
  • Mixed templates with both explicit and dynamic resources
  • Intrinsic functions work with dynamic resources
  • Conditions and DependsOn work with dynamic resources
  • Unknown resource types fall back to GenericResource
  • Model caching tests

Total: 780 tests

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR introduces dynamic resource generation from AWS CloudFormation schemas, enabling automatic creation of pycfmodel Resource classes without manual definition. The implementation downloads and caches CloudFormation schemas from AWS, generates Pydantic models with proper type hints, and automatically filters read-only properties.

Changes:

• Adds schema download and caching functionality from AWS CloudFormation schema registry
• Implements dynamic Pydantic model generation with type mapping from JSON schemas
• Provides comprehensive test coverage for CloudFormation intrinsic functions, conditions, and resource attributes

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.

File	Description
pycfmodel/schema_generator/init.py	Exports the public API for schema-based resource generation
pycfmodel/schema_generator/generator.py	Core implementation for schema downloading, parsing, and dynamic model creation
tests/schema_generator/test_generator.py	Comprehensive test suite covering intrinsic functions, conditions, metadata, and resource generation

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.