diff --git a/datastore/redis/gcp-memorystore/1.0/README.md b/datastore/redis/gcp-memorystore/1.0/README.md index a79161d5..4436540f 100644 --- a/datastore/redis/gcp-memorystore/1.0/README.md +++ b/datastore/redis/gcp-memorystore/1.0/README.md @@ -5,34 +5,225 @@ ## Overview -This module creates a managed Redis instance using Google Cloud Memorystore with high availability and security features. It integrates seamlessly with existing VPC networks and private service connections managed by the network module. +This module creates a managed Redis instance using Google Cloud Memorystore with high availability and security features. It provides enterprise-grade Redis with automatic TLS encryption, authentication, and high availability options. -The module supports both Basic and Standard HA tiers with automatic encryption, authentication, and backup capabilities. It's designed to use existing network infrastructure without creating additional private connections or IP ranges. +The module integrates with existing VPC network infrastructure through private service access, ensuring secure and isolated connectivity. It supports both development (BASIC tier) and production (STANDARD_HA tier) workloads with configurable memory, Redis versions, and security settings. ## Environment as Dimension -**Environment-aware networking**: The module scales with your environment by leveraging existing network infrastructure. Different environments will use their respective VPC networks and private service connections as configured by the network module. This ensures proper network isolation and security across development, staging, and production environments. +**Environment-aware networking**: The module scales across environments by leveraging existing network infrastructure. Each environment uses its respective VPC network and private service connections, ensuring proper network isolation and security across development, staging, and production. -The instance naming includes the environment's unique identifier to prevent conflicts and enable proper resource tracking across multiple environments. +The instance naming includes the environment's unique identifier to prevent conflicts and enable proper resource tracking across multiple environments. Security configurations like TLS can be standardized across environments or customized per environment as needed. ## Resources Created -- **Redis Memorystore Instance**: Managed Redis instance with configurable memory size and service tier -- **Authentication Configuration**: Automatic generation of secure auth tokens for Redis access -- **High Availability Setup**: Optional read replicas and regional distribution for Standard HA tier -- **Security Features**: Transit encryption and authentication enabled by default -- **Network Integration**: Private network access using existing VPC and private service connections +- **Redis Memorystore Instance**: Managed Redis with configurable memory (1-300GB) and service tier +- **Authentication Configuration**: Automatic secure auth token generation (always enabled) +- **TLS Encryption**: Configurable in-transit encryption with SERVER_AUTHENTICATION mode +- **High Availability Setup**: Read replicas and regional distribution for Standard HA tier +- **Network Integration**: Private network access via VPC private service connections +- **Certificate Management**: Automatic server CA certificate provisioning when TLS is enabled ## Security Considerations -This module implements security-first defaults that cannot be overridden: +This module implements security-first defaults designed for production workloads: -- **Private Network Access**: All instances use private IPs and existing VPC private service connections -- **Authentication Required**: Auth tokens are automatically generated and enforced -- **Transit Encryption**: Server-side authentication for encrypted connections +### Network Security +- **Private Network Access Only**: All instances use private IPs with VPC private service connections (required) - **No Public Access**: Instances are only accessible from within the configured VPC network -- **Lifecycle Protection**: Resources include prevent_destroy configuration to avoid accidental data loss +- **Network Dependencies**: Requires a VPC network module with private service access connectivity -**Network Dependencies**: This module requires a properly configured VPC network with private service access enabled. The network module must provide the private services connection and IP ranges before Redis instances can be created. +### Authentication & Encryption +- **Authentication Always Enabled**: Secure auth tokens are automatically generated and enforced +- **TLS Encryption (Default: Enabled)**: In-transit encryption using TLS 1.2+ with SERVER_AUTHENTICATION mode + - **When TLS is Enabled**: + - Port: **6378** (not the default 6379) + - Protocol: TLS 1.2 or higher only + - Mode: SERVER_AUTHENTICATION (client-to-server encryption with server authentication) + - Server CA certificates automatically provisioned and rotated + - Connection string uses `rediss://` scheme for automatic TLS client configuration + - **When TLS is Disabled** (not recommended for production): + - Port: 6379 + - No encryption + - Connection string uses `redis://` scheme +- **Important**: TLS setting **cannot be changed** after instance creation without recreating the instance -**Backup Strategy**: Memorystore provides automatic backup capabilities through GCP's infrastructure. Point-in-time recovery and backup restoration should be managed through Google Cloud Console or gcloud CLI rather than Terraform configuration. \ No newline at end of file +### Certificate Management +When TLS is enabled: +- **Server CA certificates** are automatically generated and managed by GCP +- Certificates are valid for **10 years** from creation +- New certificates become available **5 years** after creation (5-year overlap for rotation) +- Server certificate rotation occurs every **180 days** (causes brief connection drop - implement retry logic) +- CA certificates are exposed in module outputs (`attributes.server_ca_certs`) for client configuration +- Clients must download and install CA certificates from the `attributes.server_ca_certs` output + +### Data Protection +- **Lifecycle Protection**: `prevent_destroy = true` configuration prevents accidental data loss +- **Backup Strategy**: Memorystore provides automatic backup capabilities through GCP infrastructure +- **Point-in-time Recovery**: Managed through Google Cloud Console or gcloud CLI + +## TLS Performance Considerations + +### Redis Version Impact +- **Redis 7.0+**: Significantly improved TLS performance + - No connection drops during server certificate rotation + - Better encryption/decryption performance + - **Recommended for production** when using TLS +- **Redis 6.x and earlier**: May experience brief connection drops during certificate rotation + +### Connection Limits with TLS +TLS encryption introduces connection limits based on instance size: + +| Memory Tier | Redis 5.0/6.x | Redis 7.0+ | +|-------------|---------------|------------| +| M1 (1-4GB) | 1,000 | 65,000 | +| M2 (5-10GB) | 2,500 | 65,000 | +| M3 (11-35GB)| 15,000 | 65,000 | +| M4 (36-100GB)| 30,000 | 65,000 | +| M5 (101+GB) | 65,000 | 65,000 | + +**Monitor**: `redis.googleapis.com/clients/connected` metric to avoid exceeding limits. + +### Performance Optimization +When using TLS: +- Reduce connection count by reusing long-running connections +- Consider M4 or larger instance sizes for better performance +- Increase client CPU resources (compute-optimized VMs recommended) +- Reduce payload sizes to minimize encryption overhead +- Use Redis 7.0+ for optimal performance + +### Memory Impact +TLS reserves additional instance memory for encryption overhead. The `System Memory Usage Ratio` metric will be higher with TLS enabled compared to non-TLS instances. + +## Configuration + +### Redis Versions + +Supported versions (as per [GCP Memorystore documentation](https://cloud.google.com/memorystore/docs/redis/supported-versions)): +- `REDIS_7_2` - Latest version +- `REDIS_7_0` - **Default** (recommended for TLS performance) +- `REDIS_6_X` - Stable version +- `REDIS_5_0` - Legacy support + +### Service Tiers + +- **BASIC**: Standalone instance for development/testing + - Single node, no high availability + - Suitable for non-critical workloads +- **STANDARD_HA**: Highly available primary/replica instances for production + - Automatic failover + - Read replicas enabled + - Requires minimum 5GB memory + +### Security Configuration + +#### TLS Encryption Settings +- **enable_tls**: Boolean (default: `true`) + - `true`: Enables SERVER_AUTHENTICATION mode with TLS 1.2+ + - Port changes to **6378** + - Server CA certificates provided in outputs + - Connection string uses `rediss://` scheme + - `false`: No encryption (not recommended for production) + - Uses standard port 6379 + - Connection string uses `redis://` scheme + - **Cannot be changed after creation** - requires instance recreation + +### Network Requirements + +The network input module must provide: +- VPC self-link for authorized network configuration +- Private service access connection configured +- Region configuration matching the Redis instance region +- Valid private services connection details + +## Client Configuration for TLS + +When TLS is enabled (`enable_tls = true`), clients must be configured properly: + +### Required Client Setup +1. **Download CA certificates**: From module outputs (`attributes.server_ca_certs`) +2. **Configure TLS**: Point client to port **6378** (not 6379) +3. **Install certificates**: Place CA certificate file on client machine +4. **Use TLS-capable client**: Redis client library must support TLS (Redis 6.0+ for native support) + +### Connection Example +```bash +# redis-cli with TLS (requires redis-cli 6.0+) +redis-cli -h \ + -p 6378 \ + --tls \ + --cacert /path/to/server_ca.pem \ + -a \ + PING +``` + +### Alternative: Using Stunnel +For clients without native TLS support, use [Stunnel](https://www.stunnel.org/) as a TLS sidecar. + +### Connection String Format +The module automatically provides connection strings: +- **With TLS**: `rediss://:AUTH_TOKEN@HOST:6378` +- **Without TLS**: `redis://:AUTH_TOKEN@HOST:6379` + +## Module Outputs + +### Attributes +- `server_ca_certs`: Server CA certificates for TLS client configuration (sensitive, array of certificate objects, empty array if TLS disabled) +- `secrets`: List of sensitive attribute names (`["server_ca_certs"]`) + +### Interfaces +The module provides a `cluster` interface with all connection details: +- `cluster.port`: Redis connection port (6378 for TLS, 6379 for non-TLS) +- `cluster.endpoint`: Full connection endpoint in `host:port` format +- `cluster.auth_token`: Redis authentication token (sensitive, always enabled) +- `cluster.connection_string`: Ready-to-use connection string with authentication: + - With TLS: `rediss://:AUTH_TOKEN@HOST:6378` + - Without TLS: `redis://:AUTH_TOKEN@HOST:6379` +- `cluster.secrets`: List of sensitive interface fields (`["auth_token", "connection_string"]`) + +## Best Practices + +1. **Always enable TLS** for production workloads handling sensitive data +2. **Use Redis 7.0+** for better TLS performance and stability +3. **Monitor connection counts** when using TLS due to connection limits +4. **Implement retry logic** with exponential backoff for certificate rotation +5. **Use STANDARD_HA tier** for production to ensure high availability +6. **Configure private service access** before deploying the module +7. **Regularly rotate and update** CA certificates on client machines (10-year validity) +8. **Test TLS configuration** in development before deploying to production +9. **Use connection pooling** to reduce connection overhead with TLS +10. **Set appropriate memory size** accounting for TLS overhead + +## Troubleshooting TLS + +### Common Issues + +**Connection Refused**: +- Verify port is 6378 (not 6379) when TLS is enabled +- Ensure client has TLS enabled +- Check CA certificate is properly installed + +**Certificate Verification Failed**: +- Download latest CA certificates from module outputs +- Install CA on client machine +- Verify certificate file path in client configuration + +**Connection Limit Exceeded**: +- Monitor `redis.googleapis.com/clients/connected` metric +- Scale up to larger memory tier +- Implement connection pooling +- Set `timeout` config to close idle connections + +**Performance Degradation**: +- Upgrade to Redis 7.0 or higher +- Use M4 or larger instance size +- Increase client CPU resources +- Reduce payload sizes + +## References + +- [GCP Memorystore Documentation](https://cloud.google.com/memorystore/docs/redis) +- [About In-Transit Encryption](https://cloud.google.com/memorystore/docs/redis/about-in-transit-encryption) +- [Manage In-Transit Encryption](https://cloud.google.com/memorystore/docs/redis/manage-in-transit-encryption) +- [Supported Redis Versions](https://cloud.google.com/memorystore/docs/redis/supported-versions) +- [Memorystore Best Practices](https://cloud.google.com/memorystore/docs/redis/general-best-practices) diff --git a/datastore/redis/gcp-memorystore/1.0/facets.yaml b/datastore/redis/gcp-memorystore/1.0/facets.yaml index b3383629..47447a26 100644 --- a/datastore/redis/gcp-memorystore/1.0/facets.yaml +++ b/datastore/redis/gcp-memorystore/1.0/facets.yaml @@ -18,12 +18,14 @@ spec: redis_version: type: string title: Redis Version - description: Version of the Redis engine + description: Version of the Redis engine (Redis 7.0+ recommended for better + TLS performance) enum: + - REDIS_5_0 - REDIS_6_X - REDIS_7_0 - REDIS_7_2 - default: REDIS_7_2 + default: REDIS_7_0 required: - redis_version sizing: @@ -68,6 +70,17 @@ spec: - true required: - restore_from_backup + security: + type: object + title: Security Configuration + properties: + enable_tls: + type: boolean + title: Enable TLS (In-Transit Encryption) + description: Enable transit encryption (TLS) for Redis connections. When + enabled, uses SERVER_AUTHENTICATION mode with TLS 1.2+. Port will be 6378. + Cannot be changed after creation. + default: true imports: type: object title: Import Existing Resources @@ -81,6 +94,7 @@ spec: - version_config - sizing - restore_config + - security inputs: gcp_provider: type: '@facets/gcp_cloud_account' @@ -91,7 +105,7 @@ inputs: - google network: type: '@facets/gcp-network-details' - optional: true + optional: false displayName: GCP Network Configuration description: VPC and subnet configuration for the Redis instance (must have private service access enabled) @@ -117,9 +131,11 @@ sample: disabled: true spec: version_config: - redis_version: REDIS_7_2 + redis_version: REDIS_7_0 sizing: memory_size_gb: 5 tier: STANDARD_HA restore_config: restore_from_backup: false + security: + enable_tls: true diff --git a/datastore/redis/gcp-memorystore/1.0/locals.tf b/datastore/redis/gcp-memorystore/1.0/locals.tf index fad91999..cf8c9fca 100644 --- a/datastore/redis/gcp-memorystore/1.0/locals.tf +++ b/datastore/redis/gcp-memorystore/1.0/locals.tf @@ -1,26 +1,14 @@ -# Local values for GCP Redis Memorystore configuration - locals { - # Basic configuration - project_id = var.inputs.gcp_provider.attributes.project - region = var.inputs.network != null ? var.inputs.network.attributes.region : "us-central1" - vpc_name = var.inputs.network != null ? var.inputs.network.attributes.vpc_name : "default" + # GCP provider configuration + project_id = var.inputs.gcp_provider.attributes.project_id + region = var.inputs.network.attributes.region + vpc_name = var.inputs.network.attributes.vpc_name - # Instance naming - ensuring GCP naming compliance (40 chars max for Redis) - instance_name = substr( - replace( - replace( - "${var.instance_name}-${var.environment.unique_name}", - "/[^a-z0-9-]/", - "-" - ), - "/^-+|-+$/", - "" - ), - 0, 40 - ) + # Instance naming (GCP constraint: max 40 chars, lowercase alphanumeric and hyphens) + name_sanitized = lower(replace("${var.instance_name}-${var.environment.unique_name}", "/[^a-zA-Z0-9-]/", "-")) + instance_name = substr(trim(local.name_sanitized, "-"), 0, 40) - # Redis configuration + # Redis configuration from spec redis_version = var.instance.spec.version_config.redis_version memory_size_gb = var.instance.spec.sizing.memory_size_gb tier = var.instance.spec.sizing.tier @@ -29,13 +17,13 @@ locals { restore_from_backup = var.instance.spec.restore_config.restore_from_backup source_instance_id = lookup(var.instance.spec.restore_config, "source_instance_id", null) - # Network configuration - use existing private services connection from network module if available - # Fall back to default VPC for testing when network module is not available - authorized_network = var.inputs.network != null ? var.inputs.network.attributes.vpc_self_link : null + # Security and network configuration + enable_tls = var.instance.spec.security.enable_tls + authorized_network = var.inputs.network.attributes.vpc_self_link - # Redis port (standard) - redis_port = 6379 + # Note: Port is dynamically assigned by GCP (6378 with TLS, 6379 without) + # and is accessed via google_redis_instance.main.port - # Location - use first zone from the region + # Location configuration location_id = "${local.region}-a" } \ No newline at end of file diff --git a/datastore/redis/gcp-memorystore/1.0/main.tf b/datastore/redis/gcp-memorystore/1.0/main.tf index 6a592636..47ef8a77 100644 --- a/datastore/redis/gcp-memorystore/1.0/main.tf +++ b/datastore/redis/gcp-memorystore/1.0/main.tf @@ -1,18 +1,3 @@ -# Use existing private services connection from network module when available -# Falls back to default VPC for testing when network module is not provided -# The network module provides private service networking in production - -# Import existing Redis instance if specified -# Import blocks with dynamic IDs are not supported in Terraform -# The import functionality will be handled by the Facets platform -# using the import declaration in facets.yaml - -# Generate random auth string for Redis (not needed for imports - Redis generates its own) -# Removed conditional resource creation to prevent issues during import operations -# The imported Redis instance will have its own auth_string that we reference directly - -# Redis Memorystore Instance -# Uses existing private services connection provided by the network module when available resource "google_redis_instance" "main" { name = local.instance_name tier = local.tier @@ -22,26 +7,29 @@ resource "google_redis_instance" "main" { region = local.region location_id = local.location_id - # Network configuration - use existing private services connection when available - # For testing without network module, this will create in default network + # Network configuration - uses private service access from network module authorized_network = local.authorized_network - connect_mode = local.authorized_network != null ? "PRIVATE_SERVICE_ACCESS" : "DIRECT_PEERING" + connect_mode = "PRIVATE_SERVICE_ACCESS" - # Redis configuration + # Redis engine configuration redis_version = local.redis_version display_name = "Redis instance for ${var.instance_name}" - # Security configuration (hardcoded for security) - # These settings are managed by ignore_changes for imported resources - auth_enabled = true - transit_encryption_mode = local.authorized_network != null ? "SERVER_AUTHENTICATION" : "DISABLED" + # Security configuration + # AUTH is always enabled for secure access + auth_enabled = true - # High availability for standard tier - # These settings are managed by ignore_changes for imported resources + # TLS configuration (in-transit encryption) + # When enabled: Uses SERVER_AUTHENTICATION mode with TLS 1.2+, port 6378 + # When disabled: No encryption, port 6379 (not recommended for production) + # Note: Cannot be changed after instance creation (ForceNew) + transit_encryption_mode = local.enable_tls ? "SERVER_AUTHENTICATION" : "DISABLED" + + # High availability configuration (STANDARD_HA tier only) replica_count = local.tier == "STANDARD_HA" ? 1 : 0 read_replicas_mode = local.tier == "STANDARD_HA" ? "READ_REPLICAS_ENABLED" : "READ_REPLICAS_DISABLED" - # Labels for resource management + # Resource labels labels = merge( var.environment.cloud_tags, { @@ -53,47 +41,24 @@ resource "google_redis_instance" "main" { } ) - # Lifecycle management + # Lifecycle management - prevents accidental deletion and ignores external changes lifecycle { prevent_destroy = true ignore_changes = [ - # Core immutable attributes (cannot be changed after creation) - name, # Instance name cannot be changed - region, # Region cannot be changed after creation - location_id, # Location cannot be changed after creation - authorized_network, # Network configuration should not change - connect_mode, # Connection mode should not change - - # Configuration attributes that may drift or be managed externally - labels, # Ignore label changes (managed by tags) - display_name, # Ignore display name changes - transit_encryption_mode, # Ignore encryption mode changes (immutable after creation) - auth_enabled, # Ignore auth changes (should remain enabled) - replica_count, # Ignore replica count changes (managed by tier) - read_replicas_mode, # Ignore read replica mode changes (managed by tier) - - # Version can be upgraded through GCP console/CLI, ignore drift - redis_version, # Allow manual version upgrades through GCP - - # For imported resources, ignore these computed values - tier, # Service tier (imported resources keep existing) - memory_size_gb, # Memory size (imported resources keep existing) + name, + region, + location_id, + authorized_network, + connect_mode, + labels, + display_name, + transit_encryption_mode, + auth_enabled, + replica_count, + read_replicas_mode, + redis_version, + tier, + memory_size_gb, ] } - - # Restore from backup is handled by GCP's point-in-time recovery features - # This is typically done through Google Cloud Console or gcloud CLI - # rather than Terraform configuration -} - -# Note: This module is designed to work with or without a network module -# -# With network module (production): -# - Uses private service access and existing VPC from network module -# - Provides proper network isolation and security -# - Requires network module to provide private services connection -# -# Without network module (testing): -# - Falls back to default VPC with direct peering -# - Suitable for testing but not recommended for production -# - May have limited security and networking capabilities \ No newline at end of file +} \ No newline at end of file diff --git a/datastore/redis/gcp-memorystore/1.0/outputs.tf b/datastore/redis/gcp-memorystore/1.0/outputs.tf index e41da725..847439b8 100644 --- a/datastore/redis/gcp-memorystore/1.0/outputs.tf +++ b/datastore/redis/gcp-memorystore/1.0/outputs.tf @@ -1,12 +1,21 @@ locals { - output_attributes = {} + output_attributes = { + server_ca_certs = sensitive(local.enable_tls ? google_redis_instance.main.server_ca_certs : []) + secrets = ["server_ca_certs"] + } output_interfaces = { cluster = { - port = tostring(google_redis_instance.main.port) - endpoint = "${google_redis_instance.main.host}:${google_redis_instance.main.port}" - auth_token = google_redis_instance.main.auth_string - connection_string = "redis://:${google_redis_instance.main.auth_string}@${google_redis_instance.main.host}:${google_redis_instance.main.port}" - secrets = ["auth_token", "connection_string"] + port = tostring(google_redis_instance.main.port) + endpoint = "${google_redis_instance.main.host}:${google_redis_instance.main.port}" + auth_token = google_redis_instance.main.auth_string + connection_string = format( + "%s://:%s@%s:%s", + local.enable_tls ? "rediss" : "redis", + google_redis_instance.main.auth_string, + google_redis_instance.main.host, + google_redis_instance.main.port + ) + secrets = ["auth_token", "connection_string"] } } } \ No newline at end of file diff --git a/datastore/redis/gcp-memorystore/1.0/variables.tf b/datastore/redis/gcp-memorystore/1.0/variables.tf index b9dca8a0..14402385 100644 --- a/datastore/redis/gcp-memorystore/1.0/variables.tf +++ b/datastore/redis/gcp-memorystore/1.0/variables.tf @@ -1,3 +1,6 @@ +# Module configuration variables +# All variables are automatically populated by the Facets platform + variable "instance" { description = "Managed Redis instance using Google Cloud Memorystore with high availability and security" type = object({ @@ -16,6 +19,9 @@ variable "instance" { restore_from_backup = bool source_instance_id = optional(string) }) + security = object({ + enable_tls = bool + }) imports = optional(object({ instance_id = optional(string) })) @@ -47,11 +53,11 @@ variable "inputs" { type = object({ gcp_provider = object({ attributes = object({ - project = string + project_id = string credentials = string }) }) - network = optional(object({ + network = object({ attributes = object({ vpc_name = string vpc_self_link = string @@ -66,6 +72,6 @@ variable "inputs" { private_services_range_id = string private_services_range_name = string }) - })) + }) }) } \ No newline at end of file