Nextflow AWS Batch run issue

[harshaldeokarutd]

I am running Nextflow pipeline using AWS Batch, however it fails while running via AWS batch, works fine while running on local. I have added everything AWS cli, Docker and ECS container agent and created custom AMI and still it fails with the error - "no such a file or directory" (By file I mean one of the file the Nextflow pipeline needs to access to run particular process). Can anyone tell what exactly I am missing here. The Nextflow pipeline runs fine on local (i.e. when executer = "local")

Below is the error snapshot if it helps somehow to debug :

_**Caused by:
Process fastqc (4) terminated with an error exit status (127)

Command executed:

/home/ec2-user/APPS/VAC_SCRIPTS/split_qc_fastq /home/ec2-user/EXAMPLE_DATA/RNA/CtlRep1_R1.fastq.gz /home/ec2-user/EXAMPLE_DATA/RNA/CtlRep1_R2.fastq.gz CaseRep2 50000000000000 0 0 0 0 . CaseRep2".Fastqsummary.txt" 33 100

rm CaseRep2"_1_1.qc.fastq" CaseRep2"_2_1.qc.fastq"

/home/ec2-user/APPS/FastQC//fastqc -q -t 3 /home/ec2-user/EXAMPLE_DATA/RNA/CtlRep1_R1.fastq.gz /home/ec2-user/EXAMPLE_DATA/RNA/CtlRep1_R2.fastq.gz -o .

Command exit status:
127

Command output:
(empty)

Command error:
.command.sh: line 2: /home/ec2-user/APPS/VAC_SCRIPTS/split_qc_fastq: No such file or directory

Work dir:
s3://harshal-redshift-training/temp/21/dc86bf7d74fb9b6ddec8a25622e7d4**_

The file/directory '/home/ec2-user/APPS/VAC_SCRIPTS/split_qc_fastq' has all the permissions. Any help is much appreciated.

[pditommaso]

it's telling you the error

/home/ec2-user/APPS/VAC_SCRIPTS/split_qc_fastq: No such file or directory

[harshaldeokarutd]

Yeah, but it does exist at the said location, which is an ec2 instance location where if I run Nextflow pipeline locally it runs fine(without giving the mentioned error), but not when I trigger for aws batch. Am I missing something here ?