feat: add CloudFormation template for AWS Cloud compute environment IAM#1408
feat: add CloudFormation template for AWS Cloud compute environment IAM#1408stefanoboriero wants to merge 13 commits into
Conversation
Add a new "Seqera Intelligent Compute" section to the AWS Cloud compute environment docs (cloud + enterprise) describing the optional capability that offloads Nextflow tasks to a Seqera-managed Amazon ECS cluster, and the additional IAM policy required to enable it. The policy mirrors the seqera-sched compute policy: every action that AWS allows to be scoped by ARN is restricted to the seqera-sched-* prefix, with the remaining Resource "*" entries reserved for AWS APIs that don't support resource-level permissions.
The scheduler never calls DeleteSecurityGroup. The security group it manages has a static name (seqera-sched-ecs) shared across clusters in the region and is not torn down per-cluster.
Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io> Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io> Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io> Signed-off-by: Stefano Boriero <stefano.boriero@gmail.com>
Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io> Signed-off-by: Stefano Boriero <stefano.boriero@gmail.com>
…AM setup Introduces a CloudFormation template that provisions the IAM user, role, and policies required for the Seqera Platform AWS Cloud compute environment, with optional Seqera Intelligent Compute support gated by a parameter. Groups manual and programmatic IAM setup under a new "IAM resource provisioning" section, with CLI commands, parameter and output reference tables, and a collapsible template block. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
We have traditionally been a Terraform shop. Introducing CloudFormation means there is another IAC deployment method to support. Let's have a chat re: the implications on CX? |
|
While I have nothing against CloudFormation specifically, I think provisioning decisions should be driven by managed stacks that we control, and applied consistently across all IAM use-cases. This should be a platform-wide product decision, led from the web UI first, rather than added as a standalone item in the docs. For the inclusion in documentation, this page is already quite verbose extending it is most likely going to result in key parts being missed. I'd suggest moving this to a sub-page, keeping the top level to:
Finally, if we do add CloudFormation support for IAM users, it needs to be uniform across all relevant AWS resources (Batch Compute Environments, IAM Roles, etc.) rather than introduced in isolation. I would fully expect support requests for CloudFormation for customers around other areas as this will create a level of expectation. |
Sure, I feel like in person would be best to understand how CX uses Terraform and what are customers requirements, feels like discussing it here would pollute the thread |
|
About CloudFormation over Terraform, I don't think it's mutually exclusive and if it's not a too high burden on us we should strive to offer at least some level guidance on each of them, and leave the choice to the customer. I acknowledge that adding this to the official, public docs is one step higher than "some level of guidance" because it sets the expectation that this is a first class citizen of our offering, as Gavin points out in his comment. |
Makes sense, let' s first reach an agreement whether we want to actually offer this or not |
|
Hey folks, some context on this PR. This is an initial steps we plan to streamline the configuration for AWS Cloud CE especially, via a better support Cloudformation. This CE is requires much less moving parts compared to Batch, and therefore it's much simpler to be configured. The goal is to focus on AWS Cloud for now, no need to extend to other CEs (which may deprecated in the mid-term) |
Introduces a CloudFormation template that provisions the IAM user, role, and policies required for the Seqera Platform AWS Cloud compute environment, with optional Seqera Intelligent Compute support gated by a parameter. Groups manual and programmatic IAM setup under a new "IAM resource provisioning" section, with CLI commands, parameter and output reference tables, and a collapsible template block.