diff --git a/docs/modules_syntax.rst b/docs/modules_syntax.rst index 77710f51..b38a577a 100644 --- a/docs/modules_syntax.rst +++ b/docs/modules_syntax.rst @@ -7,6 +7,7 @@ :titlesonly: syntax/compose_x/ecs.details/* + syntax/compose_x/ecs.details/environment .. toctree:: diff --git a/docs/syntax/compose_x/alarms.rst b/docs/syntax/compose_x/alarms.rst index f5df65f0..3d7aed46 100644 --- a/docs/syntax/compose_x/alarms.rst +++ b/docs/syntax/compose_x/alarms.rst @@ -17,8 +17,18 @@ x-alarms Properties: {} MacroParameters: {} Settings: {} - Services: {} - Topics: [] + Services: + service-name: + Scaling: + Steps: + - LowerBound: 0 + UpperBound: 100 + Count: 2 + Topics: + - TopicArn: "arn:aws:sns:us-east-1:123456789012:my-topic" + NotifyOn: alarm + - x-sns: topic-name + NotifyOn: all Define AWS CloudWatch alarms on the different resources, services, and define actions. @@ -50,10 +60,24 @@ Services x-alarms: kafka-scaling-01: - Properties: {} + Properties: + AlarmName: "high-cpu-usage" + MetricName: "CPUUtilization" + Namespace: "AWS/ECS" + Statistic: "Average" + Period: 300 + EvaluationPeriods: 2 + Threshold: 80.0 + ComparisonOperator: "GreaterThanThreshold" Services: - : - Scaling: {} # Service scaling definition + my-service: + Scaling: + Steps: + - LowerBound: 0 + UpperBound: 100 + Count: 2 + - LowerBound: 100 + Count: 4 Scaling @@ -72,11 +96,19 @@ Topics x-alarms: alarms-01: - Properties: {} + Properties: + AlarmName: "service-health-check" + MetricName: "HealthyHostCount" + Namespace: "AWS/ApplicationELB" + Statistic: "Average" + Period: 60 + EvaluationPeriods: 2 + Threshold: 1.0 + ComparisonOperator: "LessThanThreshold" Topics: - - TopicArn: + - TopicArn: "arn:aws:sns:us-east-1:123456789012:alerts-topic" NotifyOn: okay - - x-sns: + - x-sns: notification-topic NotifyOn: all TopicArn @@ -153,7 +185,7 @@ to define is the Alarm expression .. code-block:: yaml MacroParameters: - CompositeExpression: + CompositeExpression: "ALARM(alarm-01) and ALARM(alarm-02)" ServiceName -------------- @@ -161,9 +193,24 @@ ServiceName Allows to set the service name or family defined in the compose file, and automatically fills in the ``Cluster`` and ``ServiceName`` Dimensions. +.. code-block:: yaml + + x-alarms: + service-cpu-alarm: + MacroParameters: + ServiceName: my-web-service + Properties: + MetricName: "CPUUtilization" + Namespace: "AWS/ECS" + Statistic: "Average" + Period: 300 + EvaluationPeriods: 2 + Threshold: 75.0 + ComparisonOperator: "GreaterThanThreshold" + .. note:: - If you did set ``Dimensions`` in the Properties, these will be overriden. + If you did set ``Dimensions`` in the Properties, these will be overridden. CompositeExpression -------------------- diff --git a/docs/syntax/compose_x/common.rst b/docs/syntax/compose_x/common.rst index ae2b2f26..0679955f 100644 --- a/docs/syntax/compose_x/common.rst +++ b/docs/syntax/compose_x/common.rst @@ -222,9 +222,9 @@ This is accomplished by using **AWS Resources Group Tags API** which means, you Lookup: Tags: - - Key: Value - - Key: Value - RoleArn: + - Environment: production + - Application: web-service + RoleArn: "arn:aws:iam::123456789012:role/cross-account-lookup" .. hint:: @@ -233,9 +233,30 @@ This is accomplished by using **AWS Resources Group Tags API** which means, you Tags ------ -The tags are a list of Tags that have been assigned to the resource. Based on the type of resource, this might +The tags are a list of key-value pairs that have been assigned to the resource. Based on the type of resource, this might need to resolve to a single specific resource in your AWS account / region. +Tags can be specified in two formats: + +.. code-block:: yaml + :caption: Object format (key-value pairs) + + Tags: + Environment: production + Application: web-service + Owner: devops-team + +.. code-block:: yaml + :caption: Array format (list of objects) + + Tags: + - Key: Environment + Value: production + - Key: Application + Value: web-service + - Key: Owner + Value: devops-team + RoleArn -------- @@ -247,6 +268,25 @@ your templates in a central CICD account. Compose-X will never modify the looked up object! +Example Lookup Configuration +----------------------------- + +.. code-block:: yaml + :caption: Complete lookup example + + x-s3: + existing-bucket: + Lookup: + Tags: + Environment: production + Project: myapp + RoleArn: "arn:aws:iam::123456789012:role/cross-account-lookup" + Services: + web-service: + Access: + bucket: ListOnly + objects: RW + .. warning:: @@ -257,6 +297,10 @@ your templates in a central CICD account. Tags keys and values are case sensitive. +.. note:: + + When using the object format for tags, ECS Compose-X will automatically convert them to the array format internally for AWS API compatibility. + .. _settings_syntax_reference: Settings diff --git a/docs/syntax/compose_x/ecs.details/alarms.rst b/docs/syntax/compose_x/ecs.details/alarms.rst index 4d6ce0c0..1fbdfe2f 100644 --- a/docs/syntax/compose_x/ecs.details/alarms.rst +++ b/docs/syntax/compose_x/ecs.details/alarms.rst @@ -15,7 +15,7 @@ services.x-alarms app01: x-alarms: Predefined: - RuleName: + HighCpuUsageAndMaxScaledOut: Topics: [] # Similar to other x-alarms settings Settings: {} # Input values override. @@ -65,10 +65,14 @@ count is equal to the max scaling capacity (or otherwise overriden value). .. code-block:: yaml :caption: Example at 50% CPU usage and override to 4 tasks. - - Name: HighCpuUsageAndMaxScaledOut - Settings: - CPUUtilization: 50 # In percent - RunningTaskCount: 4 # Number of tasks to evaluate against. + services: + app01: + x-alarms: + Predefined: + HighCpuUsageAndMaxScaledOut: + Settings: + CPUUtilization: 50 # In percent + RunningTaskCount: 4 # Number of tasks to evaluate against. HighRamUsageAndMaxScaledOut @@ -89,12 +93,16 @@ This rule will trigger an alert when the CPUUtilization of a given service will count is equal to the max scaling capacity (or otherwise overriden value). .. code-block:: yaml - :caption: Example at 50% CPU usage and override to 4 tasks. + :caption: Example at 50% RAM usage and override to 4 tasks. - - Name: HighRamUsageAndMaxScaledOut - Settings: - MemoryUtilization: 50 # In percent - RunningTaskCount: 4 # Number of tasks to evaluate against. + services: + app01: + x-alarms: + Predefined: + HighRamUsageAndMaxScaledOut: + Settings: + MemoryUtilization: 50 # In percent + RunningTaskCount: 4 # Number of tasks to evaluate against. diff --git a/docs/syntax/compose_x/ecs.details/ecs.rst b/docs/syntax/compose_x/ecs.details/ecs.rst index 68324d07..0b820ad9 100644 --- a/docs/syntax/compose_x/ecs.details/ecs.rst +++ b/docs/syntax/compose_x/ecs.details/ecs.rst @@ -14,10 +14,13 @@ services.x-ecs serviceA: image: nginx/nginx x-ecs: - CapacityProviderStrategy: [CapacityProviderStrategyItem] - EnableExecuteCommand: bool - CpuArchitecture: str - OperatingSystemFamily: str + CapacityProviderStrategy: + - CapacityProvider: FARGATE + Base: 1 + Weight: 2 + EnableExecuteCommand: true + CpuArchitecture: ARM64 + OperatingSystemFamily: LINUX CapacityProviderStrategy ============================ @@ -76,6 +79,7 @@ Examples - CapacityProvider: FARGATE_SPOT Base: 4 Weight: 8 + EnableExecuteCommand: true .. code-block:: yaml @@ -91,6 +95,7 @@ Examples - CapacityProvider: FARGATE Base: 1 Weight: 2 + CpuArchitecture: X86_64 nginx: deploy: labels: @@ -103,6 +108,7 @@ Examples - CapacityProvider: FARGATE_SPOT Base: 4 Weight: 8 + CpuArchitecture: X86_64 In the above example, where grafana and nginx are part of the same task definition and therefore same ECS Service, we do the following: diff --git a/docs/syntax/compose_x/ecs.details/monitoring.rst b/docs/syntax/compose_x/ecs.details/monitoring.rst index 3a7f21f3..1b687ddc 100644 --- a/docs/syntax/compose_x/ecs.details/monitoring.rst +++ b/docs/syntax/compose_x/ecs.details/monitoring.rst @@ -14,7 +14,11 @@ services.x-monitoring services: serviceA: x-monitoring: - CWAgentCollectEmf: bool|SidecarConfig + CWAgentCollectEmf: true + CollectEmf: + CloudwatchAgent: + UseLatest: true + InterpolateWithDigest: false Shorthands for monitoring features. @@ -35,7 +39,8 @@ See the `AWS CloudWatch agent & EMF Configuration for details`_ of what's config boolean value ------------- -When set to true|false, enables the sidecar using the `latest CloudWatch agent image from AWS ECR Public`_. +When set to true, enables the sidecar using the `latest CloudWatch agent image from AWS ECR Public`_. +When set to false, disables EMF collection entirely. SidecarConfig -------------- @@ -43,13 +48,15 @@ SidecarConfig This configuration allows you to define more options to control the behaviour of the used sidecar image. You must explicitly set either ``UseLatest`` or `OverrideImage`_ -.. code-block:: +.. code-block:: yaml - x-monitoring: - CWAgentCollectEmf: - InterpolateWithDigest: bool - OverrideImage: str - UseLatest: bool + services: + serviceA: + x-monitoring: + CWAgentCollectEmf: + InterpolateWithDigest: true + OverrideImage: "public.ecr.aws/cloudwatch-agent/cloudwatch-agent:1.247357.0b252275" + UseLatest: false .. hint:: @@ -58,12 +65,14 @@ You must explicitly set either ``UseLatest`` or `OverrideImage`_ OverrideImage ^^^^^^^^^^^^^^ -.. code-block:: +.. code-block:: yaml - x-monitoring: - CWAgentCollectEmf: - InterpolateWithDigest: false - OverrideImage: public.ecr.aws/cloudwatch-agent/cloudwatch-agent:1.247357.0b252275 + services: + serviceA: + x-monitoring: + CWAgentCollectEmf: + InterpolateWithDigest: false + OverrideImage: "public.ecr.aws/cloudwatch-agent/cloudwatch-agent:1.247357.0b252275" .. note:: @@ -80,12 +89,14 @@ the digest. This setting is recommended if you want to ensure that the image used is going to be consistently the same throughout the lifecycle of your Task Definition revision. -.. code-block:: +.. code-block:: yaml - x-monitoring: - CWAgentCollectEmf: - InterpolateWithDigest: true - UseLatest: true + services: + serviceA: + x-monitoring: + CWAgentCollectEmf: + InterpolateWithDigest: true + UseLatest: true .. _AWS CloudWatch agent & EMF Configuration for details: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Generation_CloudWatch_Agent.html .. _latest CloudWatch agent image from AWS ECR Public: https://gallery.ecr.aws/cloudwatch-agent/cloudwatch-agent diff --git a/docs/syntax/compose_x/ecs.details/xray.rst b/docs/syntax/compose_x/ecs.details/xray.rst index 04c2630b..52b08595 100644 --- a/docs/syntax/compose_x/ecs.details/xray.rst +++ b/docs/syntax/compose_x/ecs.details/xray.rst @@ -13,11 +13,40 @@ services.x-xray services: frontend: - x-xray: True/False + x-xray: true Automatically add the ``xray-daemon`` sidecar to your task definition, automatically defining port, environment variables for the other containers to use. +You can set this to either a boolean value or an object to customize the X-Ray daemon configuration. + +Boolean Usage +============= + +.. code-block:: yaml + + services: + frontend: + x-xray: true # Enable with default settings + + backend: + x-xray: false # Disable X-Ray + +Object Usage +============ + +.. code-block:: yaml + + services: + frontend: + x-xray: + OverrideImage: "public.ecr.aws/xray/aws-xray-daemon:3.3.7" + + +OverrideImage +============= + +When using the object format, you can specify a custom X-Ray daemon image instead of using the default AWS-provided image. IAM permissions =============== diff --git a/docs/syntax/compose_x/rds.rst b/docs/syntax/compose_x/rds.rst index d4b814ef..6641878e 100644 --- a/docs/syntax/compose_x/rds.rst +++ b/docs/syntax/compose_x/rds.rst @@ -69,14 +69,17 @@ with the ARN of the secret Services: app01: - Access: RW + Access: + DBCluster: RO app03: - Access: RW + Access: + DBCluster: RO GrantTaskAccess: SecretEnvName: DB_B_SECRET # Grants access to the secret, sets env var DB_B_SECRET to the ARN youtoo: - Access: RW - GrantTaskAccess: True # Grants access to the secret, not setting an env var + Access: + DBCluster: RO + GrantTaskAccess: true # Grants access to the secret, not setting an env var .. _rds_generate_connection_string: @@ -117,7 +120,7 @@ environment variable. EngineVersion: "5.7" BackupRetentionPeriod: 1 DatabaseName: wordpress - StorageEncrypted: True + StorageEncrypted: true Tags: - Key: Name Value: "dummy-db" @@ -235,7 +238,6 @@ might re-define in **RdsFeatures** will be skipped. If you wish to use **RdsFeat .. code-block:: yaml - :caption: Example with different bucket names syntax x-rds: dbB: @@ -283,10 +285,6 @@ It will also automatically identify the DB security group and add an ingress rul Name: cluster-identifier Tags: - sometag: value - instance: - Name: DB Instance Id - Tags: - - sometag: value secret: Tags: - sometag: value @@ -335,7 +333,7 @@ Examples dbname: Properties: Engine: aurora-mysql - EngineVersion: 5.7.12 + EngineVersion: "5.7.12" Services: app01: Access: diff --git a/docs/syntax/compose_x/tags.rst b/docs/syntax/compose_x/tags.rst index 88422109..eaf2f1fd 100644 --- a/docs/syntax/compose_x/tags.rst +++ b/docs/syntax/compose_x/tags.rst @@ -16,7 +16,9 @@ Mapping syntax :caption: Key/Value structure x-tags: - str: value + Environment: production + Application: web-server + Owner: devops-team List syntax ------------- @@ -27,15 +29,23 @@ Alternatively, you can use the default AWS CFN implementation :caption: List of Key/Value tags x-tags: - - Key: sts - Value: + - Key: Environment + Value: production + - Key: Application + Value: web-server + - Key: Owner + Value: devops-team Default tags ============= -CreatedByComposeX: true # Allows you to identify quickly if that resource was created by Compose-X -compose-x::version: # Defines which version of compose-x was used to create this resource. +The following tags are automatically added to all resources created by ECS Compose-X: + +.. code-block:: yaml + + CreatedByComposeX: "true" # Allows you to identify quickly if that resource was created by Compose-X + compose-x::version: "1.1.7" # Defines which version of compose-x was used to create this resource JSON Schema ============ diff --git a/docs/syntax/docker-compose/secrets.rst b/docs/syntax/docker-compose/secrets.rst index 21562bb9..90f74b3f 100644 --- a/docs/syntax/docker-compose/secrets.rst +++ b/docs/syntax/docker-compose/secrets.rst @@ -73,20 +73,22 @@ LinksTo Type: List of Strings -AllowedValues (also default): +AllowedValues: * EcsExecutionRole * EcsTaskRole +Default: ["EcsExecutionRole"] .. code-block:: yaml - secret-name: - x-secrets: - Name: String - LinksTo: - - EcsExecutionRole - - EcsTaskRole + secrets: + secret-name: + x-secrets: + Name: "/path/to/secret" + LinksTo: + - EcsExecutionRole + - EcsTaskRole .. warning:: @@ -105,9 +107,12 @@ Type: List of objects/dicts .. code-block:: yaml :caption: JsonKeys objects structure - SecretKey: str - VarName: str - Transform: str + JsonKeys: + - SecretKey: "username" + VarName: "DB_USER" + Transform: "capitalize" + - SecretKey: "password" + VarName: "DB_PASSWORD" SecretKey """""""""""" @@ -149,7 +154,7 @@ Examples secrets: topsecret_info: x-secrets: - Name: /path/to/my/secret + Name: "/path/to/my/secret" services: serviceA: @@ -166,7 +171,7 @@ Examples LinksTo: - EcsExecutionRole - EcsTaskRole - Name: SFTP/asl-cscs-files-dev + Name: "SFTP/asl-cscs-files-dev" .. code-block:: yaml @@ -175,20 +180,20 @@ Examples secrets: zyx: x-secrets: - Name: secret/with/kmskey + Name: "secret/with/kmskey" Lookup: Tags: - costcentre: lambda - composexdev: "yes" JsonKeys: - - SecretKey: username - VarName: PSQL_USERNAME - - SecretKey: password - VarName: PSQL_PASSWORD + - SecretKey: "username" + VarName: "PSQL_USERNAME" + - SecretKey: "password" + VarName: "PSQL_PASSWORD" .. code-block:: yaml - :caption: Secret with assignment to Task and Execution Role + :caption: Secret with assignment to Task and Execution Role using ARN secrets: abcd: {} @@ -197,7 +202,7 @@ Examples LinksTo: - EcsExecutionRole - EcsTaskRole - Name: arn:aws:secretsmanager:eu-west-1:123456789012:secret:/secret/abcd + Name: "arn:aws:secretsmanager:eu-west-1:123456789012:secret:/secret/abcd" JSON Schema ============