compilation error mystery

[davidaray]

I'm attempting to implement nextflow for some complex processes and I keep running into an error that I don't understand.

I'm testing subsections of the overall workflow and the first is TEcurate_setup.nf, which is invoked using TEcurate_setup.sh.

The error I keep getting is:

N E X T F L O W  ~  version 22.04.0
Launching `TEcurate_setup.nf` [drunk_turing] DSL2 - revision: 686f9b162f
Script compilation error
- file : /lustre/scratch/daray/ixodes/iRic1_nextflow/TEcurate_setup.nf
- cause: Unexpected character: '"' @ line 29, column 4.
    """
      ^

1 error

While I'm new to nextflow, this is the correct syntax as far as I can tell. Three " are required to establish the bash code block I'm trying to run.

What am I missing? It's probably something very simple but I'm completely missing it if that's the case.

David

[jorgeaguileraseqera]

can you share TEcurate_setup.nf file? or at least lines between 27-31

[davidaray]

Of course. See below and thank you for the quick response.

#!/usr/bin/env nextflow
 
/*
 * Defines the pipeline inputs parameters (giving a default value for each for them)
 * Each of the following parameters can be specified as command line options
 */

	params.NAME = "iRic1"
	params.MINORF = 500
	params.MINCOPY = 10
	params.WORKDIR = "/lustre/scratch/daray/ixodes/iRic1_nextflow"
	params.TARGET = "iRic1_extended_rep.fa.classified"
	params.EXTENSIONSDIR = "/lustre/scratch/daray/ixodes/iRic1_nextflow/extensions"
	params.PFAM = "/lustre/work/daray/software/pfam_db"
	params.AIDPATH = "/lustre/work/daray/software/TE-Aid"
	params.ASSEMBLIESDIR = "/lustre/scratch/daray/ixodes/assemblies"
	params.AIDOUT = "/lustre/scratch/daray/ixodes/iRic1_nextflow/te-aid"

/*
 * Call TEcurate_setup to generate the initial table and output directories.
 */

process TECURATE_SETUP { 

	output: 
	stdout into SETUP

	shell:
	"""
	#!/bin/bash
		
	sh TEcurate_setup.sh \
		params.NAME \ 
		params.MINORF \ 
		params.MINCOPY \ 
		params.WORKDIR \ 
		params.TARGET \ 
		params.EXTENSIONSDIR \ 
		params.PFAM \ 
		params.AIDPATH \ 
		params.ASSEMBLIESDIR \ 
		params.AIDOUT
	"""
	}

[glormph]

I think you need single quote marks, because of the shell: instead of script:, so ''' instead of """.
https://www.nextflow.io/docs/latest/process.html#shell

[jorgeaguileraseqera]

Problem comes because some lines have spaces after the \

for example:

sh TEcurate_setup.sh \
		params.NAME \ 

the first line it's ok but the second line ends with '\ ' and the parse fails

P.d.
I think you will need to use the '$' in the params.XXXX to interpolate their values :

	shell:
	"""
	#!/bin/bash
		
	sh TEcurate_setup.sh \
		$params.NAME \
		$params.MINORF \
		$params.MINCOPY \
		$params.WORKDIR \
		$params.TARGET \
		$params.EXTENSIONSDIR \
		$params.PFAM \
		$params.AIDPATH \
		$params.ASSEMBLIESDIR \
		$params.AIDOUT
	"""

[pditommaso]

@jorgeaguileraseqera as a sidenote, it would be nice to better report this kind of problem. I've noticed when moving from the groovy 2 to 3, it becomes more cryptic this syntax error reporting

[jorgeaguileraseqera]

with "report this kind of problem" do you mean to improve the output error of the parser?

[pditommaso]

We discussed. I was referring to a different problem

[davidaray]

Thank you for the help. I fixed the problem with the spaces. (I'd never have found that.)

Unfortunately, I'm now getting a different error:

$ nextflow run TEcurate_setup.nf
N E X T F L O W ~ version 22.04.0
Launching TEcurate_setup.nf [intergalactic_wozniak] DSL2 - revision: 29b17b2726
Missing workflow definition - DSL2 requires at least a workflow block in the main script

This is a new one.

I've run it with and without the "$" with params as suggested and get the same error either way.

[jorgeaguileraseqera]

I can see your execution is using DSL2 ( Launching TEcurate_setup.nf [intergalactic_wozniak] *DSL2*) but your script is using DSL1 syntax

it's strange because I'm running with the same version (22.04.0) and it detects it as DSL1

 ./nextflow TEcurate_setup.nf 
N E X T F L O W  ~  version 22.04.0
Launching `TEcurate_setup.nf` [prickly_shannon] DSL1 - revision: 259a52f81d
executor >  local (1)
[e7/bc1217] process > TECURATE_SETUP [100%] 1 of 1 ✔

[jorgeaguileraseqera]

can you try with

NXF_VER=22.04.2 ./nextflow run TEcurate_setup.nf

please?

[davidaray]

After downloading some dependencies, it attempted to run, which is progress. But.... see attached log file.

It appears to be a file not found error. However, the file TEcurate_setup.sh is in the current working directory and executable.

$ ls -lhrt
total 80K
drwxr-xr-- 2 daray bio 4.0K May 3 16:46 assemblies_dir
drwxr-xr-- 9 daray bio 4.0K May 3 17:59 extensions
drwxr-xr-- 2 daray bio 4.0K May 3 18:01 rmasker_dir
drwxr-xr-- 3 daray bio 4.0K May 3 18:01 rmodeler_dir
drwxr-xr-- 2 daray bio 4.0K May 5 12:09 repeatclassifier
-rwxr-xr-- 1 daray bio 8.9K May 8 07:42 TEcurate_setup.sh
-rwxr-xr-- 1 daray bio 2.4K May 8 07:42 TEcurate_nf.sh
-rwxr-xr-- 1 daray bio 2.9K May 8 07:42 TEcurate_end.sh
-rwxr-xr-- 1 daray bio 1.4K May 8 07:42 TEcurate_RC.nf
-rwxr-xr-- 1 daray bio 826 May 8 07:49 test.nf
-rwxr-xr-- 1 daray bio 372 May 8 08:04 tutorial.nf
-rwxr-xr-- 1 daray bio 2.1K May 18 08:40 TEcurate_aid2.nf
-rwxr-xr-- 1 daray bio 1.8K May 18 08:40 TEcurate_aid1.nf
-rwxr-xr-- 1 daray bio 2.3K May 18 08:42 TEcurate.nf
-rwxr-xr-- 1 daray bio 115 May 18 08:45 TEcurate.5961669.out
-rwxr-xr-- 1 daray bio 89 May 18 08:45 TEcurate.5961669.err
-rwxr-xr-- 1 daray bio 1.1K May 18 09:03 TEcurate_setup.nf
drwxr-xr-- 4 daray bio 4.0K May 18 14:18 work

nextflow.log

[jorgeaguileraseqera]

this is ok, Nextflow doesn't execute files in current directory (unless you proportionate full path to it or it is in PATH)

you can find more details about how Nextflow works in the documentation site or if you need help you can join us in the slack community channel

[davidaray]

That did the trick. I wasn't aware that I needed to provide the full path to invoke the script.

I've tested the _setup portion and it ran properly.

I'll make the same changes to the remaining processes and try them out.

Thank you for the help.