Samtools Software filling up /tmp on compute nodes

[Premas]

What would you like to see added?

By default, Samtools writes temporary files to /tmp on compute nodes. On large datasets, this can fill up the local disk and cause jobs to fail.

Recommended Solution:

Use the -T flag in Samtools commands to specify a directory ($HOME/$USER_SCRATCH) for temporary files.

Examples

samtools collate -T $USER_SCRATCH/tmp_dir...
samtools markdup -T $USER_SCRATCH/tmp_dir ...

References:

https://www.htslib.org/doc/samtools-collate.html
https://www.htslib.org/doc/samtools-markdup.html
Other Samtools commands that may need temporary file redirection: https://github.com/search?q=repo%3Asamtools%2Fsamtools%20temporary&type=code.

Where to add these changes?

https://docs.rc.uab.edu/data_management/cheaha_storage_gpfs/temporary_files/

[wwarriner]

Depending on the workflow, $LOCAL/$USER/... may be the preferred location. The researcher would have to mkdir -p ... that directory first. Mostly, this depends on file sizes! $LOCAL is limited to about 1 TB on most nodes.