output is not being staged to the publishDir

[complexgenome]

Hi there,

I've following code to run haplotype caller from GATK.
The process runs fine, the output files are generated in work directory. However, they are not staged/pushed to the publishDir as needed.

I'm unable to figure it out.

process haplotype {

        maxForks 3
        debug true
        errorStrategy 'retry'
    maxRetries 4
 label 'medium_mem'

publishDir path: "${params.outdir}/${sample_name}/haplotype/normal/", mode: 'copy', pattern: "_N*"

publishDir path: "${params.outdir}/${sample_name}/haplotype/tumor/", mode: 'copy', pattern: "_T*"

input:

        tuple val(sample_id_tumor),path(bqsrbam_tumor, stageAs: 'tumor_haplotype/*')
        tuple val(sample_id_normal),path(bqsrbam_normal, stageAs: 'normal_haplotype/*')

output:
        tuple val(sample_id_tumor), path("${sample_id_tumor}_raw.snps.indels.vcf.idx"), emit: haplotyper_tumor_idx
        tuple val(sample_id_tumor), path("${sample_id_tumor}_raw.snps.indels.vcf"), emit: haplotyper_tumor
        tuple val(sample_id_normal), path( "${sample_id_normal}_raw.snps.indels.vcf"), emit: haplotyper_normal
        tuple val(sample_id_normal), path( "${sample_id_normal}_raw.snps.indels.vcf.idx"), emit: haplotyper_normal_idx

script:

sample_name=sample_id_normal.split('_N')[0]

"""

/data1/software/gatk-4.3.0.0/gatk HaplotypeCaller -I ${bqsrbam_tumor} --intervals $params.bed_target --O "${sample_id_tumor}_raw.snps.indels.vcf" -R $params.hg38genome

/data1/software/gatk-4.3.0.0/gatk HaplotypeCaller -I ${bqsrbam_normal} --intervals $params.bed_target --O "${sample_id_normal}_raw.snps.indels.vcf" -R $params.hg38genome


"""

}

workflow.onComplete { 
        log.info ( workflow.success ? "Done haplotype caller!" : "Oops .. something went wrong in haplotype caller"  )
}

If I find outputs in the work they could be located:

find ./ -name "MM-0489_T_raw.snps.indels.vcf"
./work/0d/b7a720dcac3c4b4e7bc0a48a51ede5/MM-0489_T_raw.snps.indels.vcf

Processes are complete as shown in the attached screenshot.

[complexgenome]

@bentsherman @mribeirodantas Can you please help?

[mribeirodantas]

I deleted the repetitive tagging. Please don't do it anymore.

Can you share a minimal reproducible example so I can try running it here and debug it? Solely looking at your code, I would guess that pattern option in publishDir is not matching the output filenames.

[complexgenome]

Yes, it was pattern that caused the issue. Thank you.