AWS ECS Terraform - 1.0.0

[RobertKeyser]

This is a pretty big update with some breaking changes. It's a full overhaul, improving many aspects and modernizing it with all the latest features of both Terraform and Fides. I added a Change log, where you can see all of the actual changes, but the big things:

• Better 1:1 mapping of Terraform variables to Fides Env Vars
• Workers (and queue-specific workers too!)
• CloudFront for both Fides and the Privacy Center
• Improved Security Groups
• Allow for Private DockerHub images for Fidesplus
• Actual versioning.

[NevilleS]

Didn't review in detail (yet), just went hunting for the one specific thing I knew I'd find! 😄

[daveqnet]

@wadesdev, to try to help us in this review, here's the gemini-2.5-pro-preview-05-06 via Cursor's opinion of git diff main...HEAD on this PR branch.

This is a substantial PR with changes across multiple Terraform files. Here's a high-level overview:

New Files:

• fides-aws-ecs/s3-alb-logs.tf: Added to manage S3 bucket for Application Load Balancer (ALB) logs. This includes bucket creation, ownership controls, public access block, and a lifecycle policy for log retention.
• fides-aws-ecs/s3-dsr.tf: Added to manage an S3 bucket specifically for Data Subject Requests (DSR). This includes bucket creation, ownership controls, server-side encryption, lifecycle policy for data retention, public access block, CORS configuration, and optional bucket versioning.
• fides-aws-ecs/version.json: A new file to track the version of the fides-aws-ecs module, currently set to 1.0.0.

Modified Files:

• fides-aws-ecs/s3-privacy-center.tf:
  • The config_json_content local variable was updated to directly use privacyCenterConfig.json instead of a template file when privacy_center_configuration_file is null.
  • S3 bucket naming for privacy_center_config now incorporates var.s3_bucket_name_prefix.
  • aws_s3_bucket_acl for privacy_center_config was replaced with aws_s3_bucket_ownership_controls to enforce bucket owner enforced object ownership.
  • Dependencies for aws_s3_object resources (config_json and config_css) were updated to reflect the change from aws_s3_bucket_acl to aws_s3_bucket_ownership_controls.
• fides-aws-ecs/secrets.tf:
  • New AWS SSM Parameters were added for:
    • fides_system_scanner_pixie_api_key
    • fides_dictionary_api_key
    • fides_detection_and_discovery_website_monitor_api_key
  • New resources for Docker registry credentials:
    • aws_ssm_parameter.docker_credentials to store credentials in SSM.
    • aws_secretsmanager_secret.docker_credentials and aws_secretsmanager_secret_version.docker_credentials to store credentials in Secrets Manager for ECS.
• fides-aws-ecs/variables.tf: This file has extensive changes, introducing many new variables and modifying existing ones. Here are some key categories of changes:
  • Versioning & Image Configuration:
    • fides_version default changed from 2.3.1 to 2.60.0.
    • privacy_center_version default changed from 2.3.1 to 2.60.0.
    • New docker_credentials variable to handle private Docker image credentials.
  • Environment Configuration:
    • New environment_type variable (prod or dev).
  • Networking & DNS:
    • Validation added to fides_alternate_subnet to ensure it's different from the primary.
    • Updated description for allowed_ips.
    • Significant validation added to route53_config for subdomain and hosted zone consistency.
    • New variables custom_domain and custom_domain_points_to_cdn.
  • Fides Configuration (Many new variables):
    • Renamed/Re-scoped: fides_identity_verification and fides_require_manual_request_approval are now under a more general "Fides Execution Configuration" section.
    • New variables for CORS (fides_additional_cors_origins, fides_cors_origin_regex).
    • Extensive new variables for Fides Database, Redis, Logging, Security, Execution, Consent, Detection & Discovery, System Scanner, Celery, Dictionary, and Endpoint Cache configurations. These allow fine-grained control over various Fides components.
    • Updated fides_additional_environment_variables to include many new reserved environment variables corresponding to the new configuration options.
  • S3 Configuration:
    • New s3_bucket_name_prefix variable.
  • Postgres Configuration:
    • rds_postgres_version default changed from 13.7 to 14.17.
    • New rds_apply_immediately variable.
  • ECS Worker Configuration:
    • New workers variable to define configurations for different types of worker containers (e.g., dsr, privacy_preferences, other) with customizable CPU, memory, queue assignments, etc.
  • ALB Logs:
    • New alb_logs_retention_days variable.

Overall Impression:

This PR appears to be a major update, likely introducing significant new features and configurability to the Fides AWS ECS deployment. The changes focus on:

• Enhanced S3 Management: Dedicated buckets for ALB logs and DSR data with improved security and lifecycle management.
• Increased Configurability: A large number of new variables provide granular control over Fides core components, including database connections, Redis, logging, security policies, execution behavior, consent mechanisms, and various enterprise features (Detection & Discovery, System Scanner, Dictionary).
• Improved Security: Stricter S3 bucket policies, use of BucketOwnerEnforced for object ownership, and dedicated secrets management for Docker credentials.
• Updated Dependencies/Defaults: Newer versions for Fides, Privacy Center, and PostgreSQL.
• Modular Worker Configuration: Flexible definition of ECS worker tasks.
• Version Tracking: Introduction of version.json.

Potential Areas to Review Closely:

• Variable Defaults and Validations: Given the large number of new variables, ensure the default values are sensible and the validation rules are appropriate and cover edge cases.
• Security Implications: The new S3 buckets, secrets, and extensive configuration options have security implications. Review IAM permissions, S3 bucket policies, and access controls carefully.
• Impact of Default Version Changes: Understand the impact of upgrading Fides and PostgreSQL to the new default versions.
• Complexity: The increased configurability adds complexity. Ensure the documentation (if any in this PR or elsewhere) is updated to reflect these new options.
• Testing: Such a large change would require thorough testing across different configurations.

This is just a first pass based on the diff.