Using task.exitStatus with retries to delete large files in the failed processes work dir

[colindaven]

Dear all,

If I use the new task.exitStatus, is it possible to delete large files (eg rm *.fastq) in the work directory of a failed process?

For example following a 130 out of mem error, can I delete the half finished files easily using a closure in the work dir with the failed job?

I want to do this during the nextflow run, since workflows with many failed processes can easily sum up to 5-10 TB (so I guess afterScript is not applicable: https://www.nextflow.io/docs/latest/reference/process.html#afterscript ).

Might this idea or a variation of it work ?
afterScript = { task.exitStatus == 130 ? "rm -rf ${task.workDir}/*" : "" }

Thanks

process foo {
    memory { 2.GB * task.attempt }
    time { 1.hour * task.attempt }

    errorStrategy { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
    maxRetries 3

    script:
    <your job here>
}

[mribeirodantas]

For cleaning the work directory on the fly, please read about the nf-boost plugin here.

[bentsherman]

I have created an issue on nf-boost to track this: bentsherman/nf-boost#5

I have not considered how best to handle cleanup of failed tasks. It depends on how much information users might want to keep around for debugging