Skip to content

Is it feasible to leverage AWS Batch to parallelize the processing of individual files within a folder? #4864

@Ankita-1211

Description

@Ankita-1211

I understand that with AWS Batch as the executor, it's possible to simultaneously run various independent processes of the same workflow on different compute resources in parallel.
Is it feasible to leverage AWS Batch to parallelize the processing of individual files within a folder?
For example, if we have a FastQC process that analyzes five files in a directory, can each file be processed concurrently on separate compute instances?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions