Running offline with non-standard plugins

[fa2k]

I'm implementing the nf-core/raredisease pipeline. I've downloaded the -all version of nextflow, and downloaded the pipeline with nf-core download. This worked well until pipeline version 1.1.0, when they started using the nf-validation plugin.

How can I use nf-validation when I don't have access to the Internet?

Nexflow command:

bash-4.2$ /ess/p164/data/durable/raredisease/1.1.0/nextflow-23.06.0-edge-all run \
> /ess/p164/data/durable/raredisease/1.1.0/nf-core-raredisease-1.1.0/workflow \
> -c /ess/p164/data/durable/raredisease/1.1.0/medGenConfigs/process-overrides.conf \

[...custom configs and parameters...]

> -resume

Output:

N E X T F L O W  ~  version 23.06.0-edge
Launching `/ess/p164/data/durable/raredisease/1.1.0/nf-core-raredisease-1.1.0/workflow/main.nf` [hungry_volhard] DSL2 - revision: b28dcce5fb
WARN: Nextflow self-contained distribution allows only core plugins -- User config plugins will be ignored: nf-validation
ERROR ~ Cannot find latest version of nf-validation plugin

 -- Check script '/ess/p164/data/durable/raredisease/1.1.0/nf-core-raredisease-1.1.0/workflow/main.nf' at line: 60 or see '.nextflow.log' file for more details

[bentsherman]

Hi @fa2k , you can use the nextflow plugin command to install plugins up front:

nextflow plugin install [email protected]

This will download the plugin to $HOME/.nextflow, then you can copy that folder along with your all distribution to your offline environment.

[fa2k]

Thanks bentsherman, this is a good start, but I still have the problem. The pipeline doesn't specify the nf-validation version, so I just used: nextflow plugin install nf-validation. I get the newest version:

$ ls -l .nextflow/plugins/nf-validation-0.3.1/
total 4
drwxr-xr-x. 4 paalmbj paalmbj   38 Jul 25 14:17 classes
drwxr-xr-x. 2 paalmbj paalmbj 4096 Jul 25 14:17 lib
drwxr-xr-x. 2 paalmbj paalmbj   25 Jul 25 14:17 META-INF

After transferring the nextflow binary and $HOME/.nextflow to the offline environment, and trying again there, I still get the error.

ERROR ~ Cannot find latest version of nf-validation plugin

I also tried -offline, but no change. I've gone in and changed where it says

include {.... } from 'plugin/nf-validation'

in the workflow to

include {.... } from 'plugin/[email protected]'

but it still seems to look for the newest version and gives the same error.

[bentsherman]

You do have to pin the version if you want it to work offline. I'm not sure why it's still searching for the newest version though. Maybe you have to also pin the version in your config:

plugins {
  id '[email protected]'
}

[fa2k]

Yes! Pinning in the config works (I just didn't think about that). The modifications to the workflow I did are not necessary, and indeed they cause an error now. But with a clean workflow and this config it works. Thanks so much for the help!