What is the command used with publishDir copy

[blanleung]

Hi,

We are experiencing slowdown of our hpc head node when nextflow is copying hundreds/thousands of files from the workDirectory to the publishDirectory (publishDir set to copy). Both work and publish directories are on a nfs3 storage.
What is the command that is used by nextflow to copy files when publishDir directive is set to copy? Is that process multithreaded? If so is it possible to decrease the number of thread used to copy the files?

Many thanks!
Ben

[pditommaso]

It uses plan cp. You can try to use rclone which is expected to be more efficient setting the stageOut directive

https://www.nextflow.io/docs/latest/process.html#stageoutmode

Said that a best practice in this scenario is to run nextflow itself a cluster job, instead of running it in the head node

[blanleung]

Hi,

Thanks for your answer!

As I understand the stateOut directive is to be used with the scratch directive. However, local storage on our compute nodes is too small for the data so we keep scratch directive to false (and don't set stageOut)

I've tested your advice to run nextflow on a cluster job and indeed the head node is not affected by the slow down. However it is the compute node that suffers from slow down.

So I was wondering why the copy of files from workDir to publishDir by nextflow induces substantial slow down while copying the same files simply with cp does not.