feat: Add support for OpenTelemetry and Datadog sidecars in ECS module (#44)

- Introduced new example for X-Ray tracing with OpenTelemetry.
- Added provider configuration for AWS in the new example.
- Created variables for OpenTelemetry and Datadog configurations.
- Updated main module to include sidecar configurations for Datadog and OpenTelemetry.
- Refactored container definitions to support sidecar dependencies.
- Removed legacy container definitions and replaced with modular definitions for server, drain, scheduler, and VCS gateway.
- Enhanced secrets management to include Datadog API key and OpenTelemetry configuration content.
- Updated IAM and secrets management to accommodate new sidecar configurations.
- Improved documentation and variable descriptions for clarity.

Author: peterdeme

.spacelift/config.yml

@@ -1,5 +1,5 @@
 version: 2
-module_version: "1.6.0"
+module_version: "2.0.0"
 
 tests:
   - name: Basic example
@@ -22,5 +22,4 @@ tests:
       TF_VAR_ecr_backend_repository_url: "666640173260.dkr.ecr.eu-central-1.amazonaws.com/spacelift-backend-e89fd"
       TF_VAR_ecr_launcher_repository_url: "666640173260.dkr.ecr.eu-central-1.amazonaws.com/spacelift-launcher-e89fd"
       TF_VAR_spacelift_version: "v0.0.1-moduletests"
-      TF_VAR_license_token_wo: "<YOUR_LICENSE_TOKEN_HERE>"
-      TF_VAR_license_token_wo_version: 1
+      TF_VAR_license_token: "<YOUR_LICENSE_TOKEN_HERE>"

README.md

@@ -4,6 +4,73 @@ This module creates an ECS cluster with all the necessary resources to run Space
 
 This module is closely tied to the [terraform-aws-spacelift-selfhosted](https://github.com/spacelift-io/terraform-aws-spacelift-selfhosted) module, which contains the necessary surrounding infrastructure.
 
+> [!IMPORTANT]
+> ## 🔄 Upgrading to v2.0.0 - New features and breaking changes
+>
+> Click below to see the full upgrade guide with breaking changes and new features.
+
+<details>
+<summary><h3>📋 Full v2.0.0 Upgrade Guide</h3></summary>
+
+<br>
+
+Version 2.0.0 adds observability features and simplifies secret management. A few variables have been removed - see below for details.
+
+### ✨ New Features
+
+**Improved Secret Handling**
+- License token is now protected from state file exposure using write-only attributes
+- Secrets are automatically versioned when changed - no manual tracking needed (`license_token_wo_version` removed)
+
+**Observability & Tracing Support**
+
+This module now supports **Datadog** and **OpenTelemetry (OTEL)** sidecar containers for comprehensive application tracing and metrics:
+- **Datadog**: Full APM support with the Datadog agent sidecar
+- **OpenTelemetry**: Flexible OTEL collector sidecar that can be fully customized or configured for AWS X-Ray compatibility
+
+**Important**: AWS X-Ray SDK/Daemon enters maintenance mode on **February 25th, 2026**, with AWS limiting releases to security fixes only. AWS is transitioning to OpenTelemetry as the primary instrumentation standard for application tracing. We recommend using the OpenTelemetry sidecar option for X-Ray integration rather than the legacy X-Ray daemon.
+
+See [examples/with-datadog-tracing](./examples/with-datadog-tracing), [examples/with-xray-tracing](./examples/with-xray-tracing), and [examples/with-otel-tracing](./examples/with-otel-tracing) for complete configuration examples.
+
+### ⚠️ Breaking Changes
+
+**Removed Variables**
+- `database_url` and `database_read_only_url` (already discouraged in v1.x) - use `sensitive_env_vars` instead
+- `license_token_wo` and `license_token_wo_version` - consolidated into single `license_token` variable
+
+**Migration:**
+
+**Database URLs** (if you were still using these variables):
+```hcl
+# Before (v1.x)
+database_url           = "<connection-string>"
+database_read_only_url = "<read-only-connection-string>"
+
+# After (v2.0.0)
+sensitive_env_vars = [
+  {
+    name      = "DATABASE_URL"
+    valueFrom = "${module.spacelift_infra.database_secret_arn}:DATABASE_URL::"
+  },
+  {
+    name      = "DATABASE_READ_ONLY_URL"
+    valueFrom = "${module.spacelift_infra.database_secret_arn}:DATABASE_READ_ONLY_URL::"
+  }
+]
+```
+
+**License Token**:
+```hcl
+# Before (v1.x)
+license_token_wo         = "<your-license-token>"
+license_token_wo_version = 1  # Had to manually increment on rotation
+
+# After (v2.0.0)
+license_token = "<your-license-token>"  # Automatically versioned
+```
+
+</details>
+
 ## Module registries
 
 The module is also available [on the OpenTofu registry](https://search.opentofu.org/module/spacelift-io/ecs-spacelift-selfhosted/aws/latest) where you can browse the input and output variables.
@@ -12,7 +79,8 @@ The module is also available [on the OpenTofu registry](https://search.opentofu.
 
 Check out the [Terraform](https://developer.hashicorp.com/terraform/language/backend) or the [OpenTofu](https://opentofu.org/docs/language/settings/backends/configuration/) backend documentation for more information on how to configure the state storage.
 
-> ⚠️ Do **not** import the state into Spacelift after the installation: that would cause circular dependencies, and in case you accidentally break the Spacelift installation, you wouldn't be able to fix it.
+> [!WARNING]
+> Do **not** import the state into Spacelift after the installation: that would cause circular dependencies, and in case you accidentally break the Spacelift installation, you wouldn't be able to fix it.
 
 ## ✨ Usage
 
@@ -34,18 +102,17 @@ module "spacelift_infra" {
 }
 
 module "spacelift_services" {
-  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted?ref=v1.6.0"
+  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted"
 
   region               = local.region
   unique_suffix        = module.spacelift_infra.unique_suffix
   kms_key_arn          = module.spacelift_infra.kms_key_arn
   server_domain        = local.website_domain
   mqtt_broker_endpoint = local.mqtt_endpoint
 
-  license_token_wo = "<your-license-token-issued-by-Spacelift>" # 'wo' stands for 'write-only', it means that the token will not be stored in the state file
-  license_token_wo_version = "1" # Bump this when rotating the token
+  license_token = "<your-license-token-issued-by-Spacelift>"
 
-  encryption_type        = "kms"
+  encryption_type = "kms"
   kms_encryption_key_arn = module.spacelift_infra.kms_encryption_key_arn
   kms_signing_key_arn    = module.spacelift_infra.kms_signing_key_arn
 
@@ -115,7 +182,7 @@ You can pass in a log configuration for each service. See [the official document
 
 ```hcl
 module "spacelift_services" {
-  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted?ref=v1.6.0"
+  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted"
 
   server_log_configuration = {
     logDriver : "awslogs",
@@ -161,7 +228,7 @@ module "spacelift_services" {
 
 ```hcl
 module "spacelift_services" {
-  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted?ref=v1.6.0"
+  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted"
 
   execution_role_arn = aws_iam_role.execution_role.arn
   server_role_arn    = aws_iam_role.spacelift_server_role.arn
@@ -200,7 +267,7 @@ You will need to create a DNS record for it, then pass the following configurati
 
 ```hcl
 module "spacelift_services" {
-  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted?ref=v1.6.0"
+  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted"
 
   vcs_gateway_domain = "vcs-gateway.mycorp.io" # The DNS record for the VCS Gateway service, without protocol.
   vcs_gateway_security_group_id = module.spacelift_infra.vcs_gateway_security_group_id

examples/basic-example/providers.tf

@@ -1,12 +1,3 @@
-terraform {
-  required_providers {
-    aws = {
-      source  = "hashicorp/aws"
-      version = "< 6.0"
-    }
-  }
-}
-
 provider "aws" {
   region = var.region
 

examples/with-datadog-tracing/README.md

@@ -0,0 +1,51 @@
+# Datadog Agent Sidecar Configuration
+
+This module supports injecting a Datadog agent sidecar container into all ECS services (server, drain, scheduler, and vcs-gateway).
+
+## Usage
+
+### Basic Example
+
+```hcl
+module "spacelift" {
+  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted"
+
+  # ... other required variables ...
+
+  # Enable Datadog sidecar
+  enable_datadog_agent_sidecar = true
+  datadog_api_key              = var.datadog_api_key  # Your Datadog API key, stored in SecretsManager (versions automatically managed)
+  datadog_agent_config = {
+    image = "public.ecr.aws/datadog/agent:7"
+    site  = "datadoghq.com"  # or "datadoghq.eu" for EU
+    tags = [
+      "env:production",
+      "service:spacelift",
+      "team:platform"
+    ]
+    log_configuration = {
+      logDriver = "awslogs"
+      options = {
+        "awslogs-group"         = "/aws/ecs/datadog-agent"
+        "awslogs-region"        = var.region
+        "awslogs-create-group"  = "true"
+        "awslogs-stream-prefix" = "datadog"
+      }
+    }
+  }
+}
+```
+
+> [!IMPORTANT]
+> **Rotating Datadog API Key**
+>
+> The Datadog API key is stored in AWS Secrets Manager. To rotate it, simply update the value and apply:
+>
+> ```hcl
+> datadog_api_key = var.new_datadog_api_key
+> ```
+
+> [!TIP]
+> **Start with Logs Enabled**
+>
+> It's always recommended to start with CloudWatch logging enabled for the Datadog agent sidecar when first setting up tracing. This allows you to verify the agent is working correctly and troubleshoot any issues. Once you've confirmed everything is working as expected, you can remove the `log_configuration` to save on CloudWatch Logs costs.

examples/with-datadog-tracing/main.tf

@@ -0,0 +1,24 @@
+
+module "spacelift" {
+  source = "github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted"
+
+  # ... other required variables ...
+
+  # Observability configuration
+  enable_datadog_agent_sidecar = true
+  datadog_api_key              = var.datadog_api_key
+  datadog_agent_config = {
+    image = "public.ecr.aws/datadog/agent:7"
+    site  = var.datadog_site
+    tags  = ["env:selfhosted", "service:spacelift"]
+    log_configuration = {
+      logDriver = "awslogs"
+      options = {
+        "awslogs-group"         = "/aws/ecs/datadog-agent"
+        "awslogs-region"        = var.region
+        "awslogs-create-group"  = "true"
+        "awslogs-stream-prefix" = "datadog"
+      }
+    }
+  }
+}

examples/with-datadog-tracing/providers.tf

@@ -0,0 +1,11 @@
+provider "aws" {
+  region = var.region
+
+  default_tags {
+    tags = {
+      TfModule   = "terraform-aws-ecs-spacelift-selfhosted"
+      Repository = "https://github.com/spacelift-io/terraform-aws-ecs-spacelift-selfhosted"
+      TestCase   = "WithDatadogTracing"
+    }
+  }
+}

examples/with-datadog-tracing/variables.tf

@@ -0,0 +1,91 @@
+variable "region" {
+  type = string
+}
+
+variable "unique_suffix" {
+  type = string
+}
+
+variable "spacelift_version" {
+  type = string
+}
+
+variable "vpc_id" {
+  type = string
+}
+
+variable "public_subnet_ids" {
+  type = list(string)
+}
+
+variable "private_subnet_ids" {
+  type = list(string)
+}
+
+variable "server_security_group_id" {
+  type = string
+}
+
+variable "drain_security_group_id" {
+  type = string
+}
+
+variable "scheduler_security_group_id" {
+  type = string
+}
+
+variable "kms_key_arn" {
+  type = string
+}
+
+variable "kms_encryption_key_arn" {
+  type = string
+}
+
+variable "kms_signing_key_arn" {
+  type = string
+}
+
+variable "server_lb_name" {
+  type        = string
+  description = "The name of the server load balancer."
+  default     = null
+}
+
+variable "lb_certificate_arn" {
+  type = string
+}
+
+variable "website_domain" {
+  type = string
+}
+
+variable "mqtt_broker_endpoint" {
+  type = string
+}
+
+variable "ecr_backend_repository_url" {
+  type = string
+}
+
+variable "ecr_launcher_repository_url" {
+  type = string
+}
+
+variable "load_balancer_security_group_id" {
+  type        = string
+  default     = null
+  description = "The security group ID to use for the main load balancer. If not provided, a new security group will be created."
+}
+
+variable "datadog_api_key" {
+  type        = string
+  description = "The Datadog API key."
+  sensitive   = true
+}
+
+variable "datadog_site" {
+  type        = string
+  description = "The Datadog site to send metrics and traces to."
+  default     = "datadoghq.com"
+}