Is it possible to run multiple versions of a pipeline in parallel?

[AJnsm]

I want to run two different versions of a single pipeline because I want to run the latest version as well as continue an old run with the -resume flag.

I first pulled both

nextflow pull {name of pipeline} -r 520967e399
nextflow pull {name of pipeline} -r develop

And then ran (in parallel screen sessions and different directories):

nextflow run {name of pipeline} -r 520967e399 -resume

and

nextflow run {name of pipeline} -r develop

I assumed that this would run one instance of the latest version on the develop branch, and one of the pipeline as it was at the specified commit. Instead, the first one (with the specified commit) crashed with exit status 2, because it included an argument that was removed in the latest version of the pipeline. Apparently, specifying the commit did not really work as I expected. It looks like the latest pulled version of the pipeline overwrites all others, is this true? How should one do this instead?

[AJnsm]

Ok I found the issue: when running a specific revision, the git HEAD is moved to the appropriate place across all parallel screen sessions, which changes the version across all runs and overrides the revision flag. A possible solution if you need parallel runs of different versions would be to set NXF_HOME to the project home directory in each parallel screen session.

[ammaraziz]

Thanks for coming back to answer your own question. I think we have very similar setups, with screen + nextflow.

Appreciate it! Very helpful.