Conditional statement that runs Nextflow process only when filename contains specific substring

[adelinaLCH]

I am trying to implement conditional statements in Nextflow and I am struggling to get my pipeline to work. Essentially, when I start running my Nextflow pipeline, I have multiple files that go through each process. Files that contain the word 'control' in them need to be processed slightly different from files that do not.

In the example below, I want all my read pairs to run through trim_fastq, but if a read pair contains the word 'control' in its filename, skip running bwa on that file.

This is my primary_process.nf script:

#!/usr/bin/env nextflow

process trim_fastq {

        publishDir "$params.outdir/trimmed_fastq", mode: 'copy'
        tag "$sample"

        input:
                tuple val(sample), path(read1), path(read2) 
        output:
                tuple val(sample), path('*_trimmed.fastq.gz'), emit: trimmed_files_ch 
        script:
        """
                fastp -i ${read1} -I ${read2} -o ${sample}_R1_trimmed.fastq.gz -O ${sample}_R2_trimmed.fastq.gz
        """
}

process bwa {

        publishDir "$params.outdir/bam", mode: 'copy'
        tag "$sample"

        input:
                tuple val(sample), path(reads) 

        output:
                tuple val(sample), path("${sample}_trim_align.txt"), emit: bam_ch, optional: true

        script:
        """
                if [[ ${sample}=~*"control"* ]];
                then
                        echo "Testing bwa process." >> "${sample}_trim_align.txt"
                fi
        """
}

This is my main.nf script:

nextflow.enable.dsl=2

include { trim_fastq; bwa} from './primary_process.nf'

read_pairs = Channel.fromFilePairs(params.fastq_files, flat: true)

workflow {

        //PRIMARY PIPELINE
        trimmed_files_ch = trim_fastq(read_pairs)
        bwa(trim_fastq.out.trimmed_files_ch)
}

What is happening right now is that Nextflow ignores the if statement and runs all the read pairs through bwa, even if they contain the word 'control'.

This is the command I use to fire up running the pipeline. The fastq_files flag points to a bucket that contains paired end reads for two samples, one with the word 'control' in the filename and one without.

nextflow run main.nf --fastq_files 's3://resources/fastq_gz_read_pairs/*_R{1,2}.fastq.gz' --outdir s3://output/results/

So s3://resources/fastq_gz_read_pairs/ contains something like this:
control_R1.fastq.gz
control_R2.fastq.gz
sample_R1.fastq.gz
sample_R2.fastq.gz

I need help in getting my if statement in the bwa process to work. Thanks!

[adelinaLCH]

I figured it out. In the bwa process...

If you want the conditional statement to only run for filenames that contain the word "control" in the filename:

    publishDir "$params.outdir/bam", mode: 'copy'
    tag "$sample"

    input:
            tuple val(sample), path(reads) 

    output:
            tuple val(sample), path("${sample}_trim_align.txt"), emit: bam_ch, optional: true

    script:
    """
            if [[ "${sample}" =~ .*control.* ]];
            then
                    echo "Testing bwa process." >> "${sample}_trim_align.txt"
            fi
    """

If you want the conditional statement to only run for filenames that DO NOT contain the word "control" in the filename:

    publishDir "$params.outdir/bam", mode: 'copy'
    tag "$sample"

    input:
            tuple val(sample), path(reads) 

    output:
            tuple val(sample), path("${sample}_trim_align.txt"), emit: bam_ch, optional: true

    script:
    """
            if [[ ! "${sample}" =~ .*control.* ]];
            then
                    echo "Testing bwa process." >> "${sample}_trim_align.txt"
            fi
    """

[bentsherman]

Hi @adelinaLCH , the best way to implement conditional logic like this is with channel operators. In this case, you can use the filter operator to filter the samples based on a condition. For example:

workflow {
        trimmed_files_ch = trim_fastq(read_pairs)
        trimmed_files_ch
          | filter { sample, reads -> !sample.contains("control") }
          | bwa
}

You could also use the branch operator to route samples to one process or another based on a condition. This approach is better because it won't spawn a bunch of no-op tasks.