Skip to content

Commit 0ddd2eb

Browse files
adamrtalbotjustinegeffen
adamrtalbot
and
justinegeffen
authored
docs: Update manual AWS Batch launch template scripts (#993)
Update the launch template userdata scripts in the manual AWS Batch compute environment documentation to match the current Batch Forge implementation. Changes include: - Add logging redirection for troubleshooting - Update CloudWatch agent download from wget to curl - Update CloudWatch config from custom-v0.1.json to config-v0.4.json - Add 5 new ECS settings (Spot draining, container timeouts) - Update miniconda tarball URL to versioned package - Update sed placeholder from custom-id to $FORGE_ID - Add section comments for better readability 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Justine Geffen <justinegeffen@users.noreply.github.com>
1 parent f06bfc8 commit 0ddd2eb

File tree

5 files changed

+205
-90
lines changed

5 files changed

+205
-90
lines changed

‎platform-cloud/docs/enterprise/advanced-topics/manual-aws-batch-setup.mdx‎

Lines changed: 38 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -127,18 +127,22 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
127127
owner: root
128128
content: |
129129
#!/usr/bin/env bash
130+
exec > >(tee /var/log/tower-forge.log|logger -t TowerForge -s 2>/dev/console) 2>&1
131+
##
130132
yum install -q -y jq sed wget unzip nvme-cli lvm2
131-
wget -q https://amazoncloudwatch-agent.s3.amazonaws.com/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
133+
## install CloudWatch agent
134+
curl -s https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm -o amazon-cloudwatch-agent.rpm
132135
rpm -U ./amazon-cloudwatch-agent.rpm
133136
rm -f ./amazon-cloudwatch-agent.rpm
134-
curl -s https://nf-xpack.seqera.io/amazon-cloudwatch-agent/custom-v0.1.json \
135-
# | sed 's/custom-id/<your custom ID>/g' \
137+
curl -s https://nf-xpack.seqera.io/amazon-cloudwatch-agent/config-v0.4.json \
138+
# | sed 's/$FORGE_ID/<your custom ID>/g' \
136139
> /opt/aws/amazon-cloudwatch-agent/bin/config.json
137140
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
138141
-a fetch-config \
139142
-m ec2 \
140143
-s \
141144
-c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
145+
## format NVMe storage for Fusion
142146
mkdir -p /scratch/fusion
143147
NVME_DISKS=($(nvme list | grep 'Amazon EC2 NVMe Instance Storage' | awk '{ print $1 }'))
144148
NUM_DISKS=${#NVME_DISKS[@]}
@@ -155,18 +159,26 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
155159
fi
156160
fi
157161
chmod a+w /scratch/fusion
162+
## ECS configuration
158163
mkdir -p /etc/ecs
159164
echo ECS_IMAGE_PULL_BEHAVIOR=once >> /etc/ecs/ecs.config
160165
echo ECS_ENABLE_AWSLOGS_EXECUTIONROLE_OVERRIDE=true >> /etc/ecs/ecs.config
166+
echo ECS_ENABLE_SPOT_INSTANCE_DRAINING=true >> /etc/ecs/ecs.config
167+
echo ECS_CONTAINER_CREATE_TIMEOUT=10m >> /etc/ecs/ecs.config
168+
echo ECS_CONTAINER_START_TIMEOUT=10m >> /etc/ecs/ecs.config
169+
echo ECS_CONTAINER_STOP_TIMEOUT=10m >> /etc/ecs/ecs.config
170+
echo ECS_MANIFEST_PULL_TIMEOUT=10m >> /etc/ecs/ecs.config
171+
## stop docker
161172
systemctl stop docker
162173
## install AWS CLI
163-
mkdir -p /home/ec2-user
164-
curl -s https://nf-xpack.seqera.io/miniconda-awscli/miniconda-awscli.tar.gz \
165-
| tar xz -C /home/ec2-user
174+
curl -s https://nf-xpack.seqera.io/miniconda-awscli/miniconda-25.3.1-awscli-1.40.12.tar.gz \
175+
| tar xz -C /
166176
export PATH=$PATH:/home/ec2-user/miniconda/bin
167177
ln -s /home/ec2-user/miniconda/bin/aws /usr/bin/aws
178+
## restart docker
168179
systemctl start docker
169180
systemctl enable --now --no-block ecs
181+
## kernel settings to prevent OOM
170182
echo "1258291200" > /proc/sys/vm/dirty_bytes
171183
echo "629145600" > /proc/sys/vm/dirty_background_bytes
172184

@@ -176,7 +188,7 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
176188
--//--
177189
```
178190

179-
1. To prepend a custom identifier to the CloudWatch log streams for AWS resources created by your manual compute environment, uncomment the `| sed 's/custom-id/<your custom ID>/g' \` line and replace `<your custom ID>` with your custom ID. If ommitted, this defaults to `custom-id`.
191+
1. To prepend a custom identifier to the CloudWatch log streams for AWS resources created by your manual compute environment, uncomment the `| sed 's/$FORGE_ID/<your custom ID>/g' \` line and replace `<your custom ID>` with your custom identifier. If omitted, `$FORGE_ID` remains as-is in the config.
180192
1. Save the template with the name `seqera-launchtemplate`.
181193

182194
</TabItem>
@@ -199,30 +211,41 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
199211
owner: root
200212
content: |
201213
#!/usr/bin/env bash
214+
exec > >(tee /var/log/tower-forge.log|logger -t TowerForge -s 2>/dev/console) 2>&1
215+
##
202216
yum install -q -y jq sed wget unzip nvme-cli lvm2
203-
wget -q https://amazoncloudwatch-agent.s3.amazonaws.com/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
217+
## install CloudWatch agent
218+
curl -s https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm -o amazon-cloudwatch-agent.rpm
204219
rpm -U ./amazon-cloudwatch-agent.rpm
205220
rm -f ./amazon-cloudwatch-agent.rpm
206-
curl -s https://nf-xpack.seqera.io/amazon-cloudwatch-agent/custom-v0.1.json \
207-
# | sed 's/custom-id/<your custom ID>/g' \
221+
curl -s https://nf-xpack.seqera.io/amazon-cloudwatch-agent/config-v0.4.json \
222+
# | sed 's/$FORGE_ID/<your custom ID>/g' \
208223
> /opt/aws/amazon-cloudwatch-agent/bin/config.json
209224
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
210225
-a fetch-config \
211226
-m ec2 \
212227
-s \
213228
-c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
229+
## ECS configuration
214230
mkdir -p /etc/ecs
215231
echo ECS_IMAGE_PULL_BEHAVIOR=once >> /etc/ecs/ecs.config
216232
echo ECS_ENABLE_AWSLOGS_EXECUTIONROLE_OVERRIDE=true >> /etc/ecs/ecs.config
233+
echo ECS_ENABLE_SPOT_INSTANCE_DRAINING=true >> /etc/ecs/ecs.config
234+
echo ECS_CONTAINER_CREATE_TIMEOUT=10m >> /etc/ecs/ecs.config
235+
echo ECS_CONTAINER_START_TIMEOUT=10m >> /etc/ecs/ecs.config
236+
echo ECS_CONTAINER_STOP_TIMEOUT=10m >> /etc/ecs/ecs.config
237+
echo ECS_MANIFEST_PULL_TIMEOUT=10m >> /etc/ecs/ecs.config
238+
## stop docker
217239
systemctl stop docker
218-
## install AWS CLI v2
219-
mkdir -p /home/ec2-user
220-
curl -s https://nf-xpack.seqera.io/miniconda-awscli/miniconda-awscli.tar.gz \
221-
| tar xz -C /home/ec2-user
240+
## install AWS CLI
241+
curl -s https://nf-xpack.seqera.io/miniconda-awscli/miniconda-25.3.1-awscli-1.40.12.tar.gz \
242+
| tar xz -C /
222243
export PATH=$PATH:/home/ec2-user/miniconda/bin
223244
ln -s /home/ec2-user/miniconda/bin/aws /usr/bin/aws
245+
## restart docker
224246
systemctl start docker
225247
systemctl enable --now --no-block ecs
248+
## kernel settings to prevent OOM
226249
echo "1258291200" > /proc/sys/vm/dirty_bytes
227250
echo "629145600" > /proc/sys/vm/dirty_background_bytes
228251

@@ -232,7 +255,7 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
232255
--//--
233256
```
234257

235-
1. To prepend a custom identifier to the CloudWatch log streams for AWS resources created by your manual compute environment, uncomment the `| sed 's/custom-id/<your custom ID>/g' \` line and replace `<your custom ID>` with your custom ID. If ommitted, this defaults to `custom-id`.
258+
1. To prepend a custom identifier to the CloudWatch log streams for AWS resources created by your manual compute environment, uncomment the `| sed 's/$FORGE_ID/<your custom ID>/g' \` line and replace `<your custom ID>` with your custom identifier. If omitted, `$FORGE_ID` remains as-is in the config.
236259
1. Save the template with the name `seqera-launchtemplate`.
237260

238261
</TabItem>

‎platform-enterprise_docs/enterprise/advanced-topics/manual-aws-batch-setup.mdx‎

Lines changed: 43 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -163,18 +163,22 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
163163
owner: root
164164
content: |
165165
#!/usr/bin/env bash
166+
exec > >(tee /var/log/tower-forge.log|logger -t TowerForge -s 2>/dev/console) 2>&1
167+
##
166168
yum install -q -y jq sed wget unzip nvme-cli lvm2
167-
wget -q https://amazoncloudwatch-agent.s3.amazonaws.com/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
169+
## install CloudWatch agent
170+
curl -s https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm -o amazon-cloudwatch-agent.rpm
168171
rpm -U ./amazon-cloudwatch-agent.rpm
169172
rm -f ./amazon-cloudwatch-agent.rpm
170-
curl -s https://nf-xpack.seqera.io/amazon-cloudwatch-agent/custom-v0.1.json \
171-
# | sed 's/custom-id/<your custom ID>/g' \
173+
curl -s https://nf-xpack.seqera.io/amazon-cloudwatch-agent/config-v0.4.json \
174+
# | sed 's/$FORGE_ID/<your custom ID>/g' \
172175
> /opt/aws/amazon-cloudwatch-agent/bin/config.json
173176
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
174177
-a fetch-config \
175178
-m ec2 \
176179
-s \
177-
-c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
180+
-c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
181+
## format NVMe storage for Fusion
178182
mkdir -p /scratch/fusion
179183
NVME_DISKS=($(nvme list | grep 'Amazon EC2 NVMe Instance Storage' | awk '{ print $1 }'))
180184
NUM_DISKS=${#NVME_DISKS[@]}
@@ -191,18 +195,26 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
191195
fi
192196
fi
193197
chmod a+w /scratch/fusion
198+
## ECS configuration
194199
mkdir -p /etc/ecs
195200
echo ECS_IMAGE_PULL_BEHAVIOR=once >> /etc/ecs/ecs.config
196201
echo ECS_ENABLE_AWSLOGS_EXECUTIONROLE_OVERRIDE=true >> /etc/ecs/ecs.config
202+
echo ECS_ENABLE_SPOT_INSTANCE_DRAINING=true >> /etc/ecs/ecs.config
203+
echo ECS_CONTAINER_CREATE_TIMEOUT=10m >> /etc/ecs/ecs.config
204+
echo ECS_CONTAINER_START_TIMEOUT=10m >> /etc/ecs/ecs.config
205+
echo ECS_CONTAINER_STOP_TIMEOUT=10m >> /etc/ecs/ecs.config
206+
echo ECS_MANIFEST_PULL_TIMEOUT=10m >> /etc/ecs/ecs.config
207+
## stop docker
197208
systemctl stop docker
198209
## install AWS CLI
199-
mkdir -p /home/ec2-user
200-
curl -s https://nf-xpack.seqera.io/miniconda-awscli/miniconda-awscli.tar.gz \
201-
| tar xz -C /home/ec2-user
210+
curl -s https://nf-xpack.seqera.io/miniconda-awscli/miniconda-25.3.1-awscli-1.40.12.tar.gz \
211+
| tar xz -C /
202212
export PATH=$PATH:/home/ec2-user/miniconda/bin
203213
ln -s /home/ec2-user/miniconda/bin/aws /usr/bin/aws
214+
## restart docker
204215
systemctl start docker
205216
systemctl enable --now --no-block ecs
217+
## kernel settings to prevent OOM
206218
echo "1258291200" > /proc/sys/vm/dirty_bytes
207219
echo "629145600" > /proc/sys/vm/dirty_background_bytes
208220

@@ -212,7 +224,7 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
212224
--//--
213225
```
214226

215-
1. To prepend a custom identifier to the CloudWatch log streams for AWS resources created by your manual compute environment, uncomment the `| sed 's/custom-id/<your custom ID>/g' \` line and replace `<your custom ID>` with your custom ID. If ommitted, this defaults to `custom-id`.
227+
1. To prepend a custom identifier to the CloudWatch log streams for AWS resources created by your manual compute environment, uncomment the `| sed 's/$FORGE_ID/<your custom ID>/g' \` line and replace `<your custom ID>` with your custom identifier. If omitted, `$FORGE_ID` remains as-is in the config.
216228
1. Save the template with the name `seqera-launchtemplate`.
217229

218230
</TabItem>
@@ -235,30 +247,41 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
235247
owner: root
236248
content: |
237249
#!/usr/bin/env bash
250+
exec > >(tee /var/log/tower-forge.log|logger -t TowerForge -s 2>/dev/console) 2>&1
251+
##
238252
yum install -q -y jq sed wget unzip nvme-cli lvm2
239-
wget -q https://amazoncloudwatch-agent.s3.amazonaws.com/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
253+
## install CloudWatch agent
254+
curl -s https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm -o amazon-cloudwatch-agent.rpm
240255
rpm -U ./amazon-cloudwatch-agent.rpm
241256
rm -f ./amazon-cloudwatch-agent.rpm
242-
curl -s https://nf-xpack.seqera.io/amazon-cloudwatch-agent/custom-v0.1.json \
243-
# | sed 's/custom-id/<your custom ID>/g' \
257+
curl -s https://nf-xpack.seqera.io/amazon-cloudwatch-agent/config-v0.4.json \
258+
# | sed 's/$FORGE_ID/<your custom ID>/g' \
244259
> /opt/aws/amazon-cloudwatch-agent/bin/config.json
245260
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
246261
-a fetch-config \
247262
-m ec2 \
248263
-s \
249264
-c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
265+
## ECS configuration
250266
mkdir -p /etc/ecs
251267
echo ECS_IMAGE_PULL_BEHAVIOR=once >> /etc/ecs/ecs.config
252268
echo ECS_ENABLE_AWSLOGS_EXECUTIONROLE_OVERRIDE=true >> /etc/ecs/ecs.config
269+
echo ECS_ENABLE_SPOT_INSTANCE_DRAINING=true >> /etc/ecs/ecs.config
270+
echo ECS_CONTAINER_CREATE_TIMEOUT=10m >> /etc/ecs/ecs.config
271+
echo ECS_CONTAINER_START_TIMEOUT=10m >> /etc/ecs/ecs.config
272+
echo ECS_CONTAINER_STOP_TIMEOUT=10m >> /etc/ecs/ecs.config
273+
echo ECS_MANIFEST_PULL_TIMEOUT=10m >> /etc/ecs/ecs.config
274+
## stop docker
253275
systemctl stop docker
254-
## install AWS CLI v2
255-
mkdir -p /home/ec2-user
256-
curl -s https://nf-xpack.seqera.io/miniconda-awscli/miniconda-awscli.tar.gz \
257-
| tar xz -C /home/ec2-user
276+
## install AWS CLI
277+
curl -s https://nf-xpack.seqera.io/miniconda-awscli/miniconda-25.3.1-awscli-1.40.12.tar.gz \
278+
| tar xz -C /
258279
export PATH=$PATH:/home/ec2-user/miniconda/bin
259280
ln -s /home/ec2-user/miniconda/bin/aws /usr/bin/aws
281+
## restart docker
260282
systemctl start docker
261283
systemctl enable --now --no-block ecs
284+
## kernel settings to prevent OOM
262285
echo "1258291200" > /proc/sys/vm/dirty_bytes
263286
echo "629145600" > /proc/sys/vm/dirty_background_bytes
264287

@@ -268,15 +291,15 @@ Create a launch template to configure the EC2 instances deployed by Batch jobs:
268291
--//--
269292
```
270293

271-
1. To prepend a custom identifier to the CloudWatch log streams for AWS resources created by your manual compute environment, uncomment the `| sed 's/custom-id/<your custom ID>/g' \` line and replace `<your custom ID>` with your custom ID. If ommitted, this defaults to `custom-id`.
294+
1. To prepend a custom identifier to the CloudWatch log streams for AWS resources created by your manual compute environment, uncomment the `| sed 's/$FORGE_ID/<your custom ID>/g' \` line and replace `<your custom ID>` with your custom identifier. If omitted, `$FORGE_ID` remains as-is in the config.
272295
1. Save the template with the name `seqera-launchtemplate`.
273296

274297
</TabItem>
275298
</Tabs>
276299

277300
### Create the Batch compute environments
278301

279-
:::caution
302+
:::caution
280303
AWS Graviton instances (ARM64 CPU architecture) are not supported in manual compute environments. To use Graviton instances, create your AWS Batch compute environment with [Batch Forge](../../compute-envs/aws-batch#automatic-configuration-of-batch-resources).
281304
:::
282305

@@ -310,7 +333,7 @@ The head queue requires an on-demand compute environment. Do not select **Use Sp
310333
:::
311334
1. Expand **Additional configuration** and select the `seqera-launchtemplate` from the Launch template dropdown.
312335
1. Configure VPCs, subnets, and security groups on the next page as needed.
313-
1. Review your configuration and select **Create compute environment**.
336+
1. Review your configuration and select **Create compute environment**.
314337

315338
</TabItem>
316339
<TabItem value="Compute queue with Spot instances" label="Compute queue with Spot instances" default>
@@ -320,11 +343,11 @@ Create this compute environment to use Spot instances for your workflow compute
320343
1. In the [Batch Console](https://eu-west-1.console.aws.amazon.com/batch/home), select **Create** on the Compute environments page.
321344
1. Select **Amazon EC2** as the compute environment configuration.
322345
1. Enter a name of your choice, and apply the `seqera-servicerole` and `seqera-instancerole`.
323-
1. Select **Enable using Spot instances** to use Spot instances and save computing costs.
346+
1. Select **Enable using Spot instances** to use Spot instances and save computing costs.
324347
1. Select the `seqera-fleetrole` and enter vCPU limits and instance types, if needed.
325348
1. Expand **Additional configuration** and select the `seqera-launchtemplate` from the Launch template dropdown.
326349
1. Configure VPCs, subnets, and security groups on the next page as needed.
327-
1. Review your configuration and select **Create compute environment**.
350+
1. Review your configuration and select **Create compute environment**.
328351

329352
</TabItem>
330353
</Tabs>

0 commit comments

Comments
 (0)