how to understand file and into command

[asmlgkj]

process splitLetters {

output:
file 'chunk_*' into letters

"""
printf '${params.str}' | split -b 6 - chunk_
"""

}

how should I understand the file and into command, I tried to find such grammar in groove，but failed.
and how to understand the chunk_ and chunk_*

another thing, is there any recommended reference to learn groove for better usage of nextflow, thanks a lot

process convertToUpper {

input:
file x from letters

output:
stdout result

"""
rev $x
"""

}

[mribeirodantas]

file, into and from are part of Nextflow, a Domain Specific Language built on top of Groovy. That's why you did not find it in Groovy documentation. You should look for it in the official documentation of Nextflow. However, I must warn you, these are deprecated now since recently we moved to DSL2 as the default.

You should now use path instead of file, and use the workflow function instead of using the into/from. The * in chunck_* is a blob, capture everything that starts with chunck_. Knowing groovy is useful, but you should focus to learn Nextflow instead. For Groovy, check here.

[asmlgkj]

@mribeirodantas
Thanks a lot for your kind help.
it is still hard for me to know which has been deprecated in DSL2.
the doc is a full doc https://www.nextflow.io/docs/latest/index.html

though it has a DSL2 part, but it is stiil very hard for me to know which part has been deprecated since this does not mention (file, iinto ) have been deprecated .
if I want to learn DSL2, I see the link https://www.nextflow.io/docs/latest/dsl2.html , but it does not mention basic operator and other concepts, I still need to read the doc from beginning, it is hard to choose.
I can still see **from** in DSL part 
though the last part say, Operator print and println have been deprecated by DSL2 syntax. Use [view](https://www.nextflow.io/docs/latest/operator.html#operator-view) instead. but println still appeared 

does DLS2 support the other blocks except shell block ?
the following command output not expected

N E X T F L O W ~ version 21.10.6
Launching n4.nf [cheeky_pesquet] - revision: f31c302a94
executor > local (3)
[9c/bdf6cf] process > myTask (3) [100%] 3 of 3 ✔

when add nextflow.enable.dsl=2 in the header
also output not expected
N E X T F L O W ~ version 21.10.6
Launching n4.nf [marvelous_coulomb] - revision: 6907019a8f

process myTask {
    input:
    val str from 'Hello', 'Hola', 'Bonjour'

    shell:
    '''
    echo User $USER says !{str}
    '''
}

[mribeirodantas]

There are two different things. Some things are deprecated regardless of DSL1 or DSL2, which means they may still work but we suggest users to refrain from using them, because there are better ways. And another thing is when things are deprecated because they will only work in Nextflow DSL2, which means you can still use them but you have to set DSL1 (either export NXF_DEFAULT_DSL=1, set nextflow.enable.dsl = 1 at the top of your pipeline script or use the command line option -dsl1. I think it's important to have both things clear in mind so I hope I'm not being too annoying explaining this 😅

We've been working hard to update all the training material and documentation to DSL2 and non-deprecated info but unfortunately there are still some snippets where deprecated/DSL1 code can be found. For the from channel factory specifically, you can see the warning here.

As for material on how to "migrate" to DSL2, there are some nice links: 1, 2, 3, 4 and 5.

As for your snippet, it runs just fine here. If you add the nextflow.enable.dsl=2, I got no error but a warning that there is no workflow function. Then I ran this and it worked fine:

nextflow.enable.dsl=2
process myTask {
    input:
    val str

    shell:
    '''
    echo User $USER says !{str}
    '''
}

workflow {
    myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

If you got an error or something different from what I got, please share with me your version of Nextflow so I can try to reproduce (nextflow -version).

[asmlgkj]

@mribeirodantas
Thanks a lot for your hard work。
I'm just here to figure out how to learn better, because it feels a little expensive to learn. I want to transfer our pipeline to nextflow.This tool is very powerful。
Q1: Is there a least part for a nextflow script to run? like a process can not run, must depends on workflow.

nextflow.enable.dsl=2
process myTask {
    input:
    val str

    shell:
    '''
    echo User $USER says !{str}
    '''
}

workflow {
    myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

outout is:
N E X T F L O W ~ version 21.10.6
Launching n4.nf [elegant_poincare] - revision: 4a74c45132
executor > local (3)
[be/27eb67] process > myTask (1) [100%] 3 of 3 ✔

it does not print Hello', 'Hola', 'Bonjour one by one in the screen
The shell block; does it support python or R block ? 
is there a plan to supplort Chinese cloud, like aliyun, huawei

[mribeirodantas]

Q1: For DSL2, you don't have from/into so you must tell Nextflow how to handle the processes. That's why you need the workflow part.

Q2: You have to run with -process.echo in order to see the printing 😄

I don't know if there are official plans to support Chinese cloud providers.

[asmlgkj]

thanks a lot.it works.
But I do not know what -process.echo mean? what is the grammer here?
does this block support python, R

[mribeirodantas]

It allows your pipeline to write to stdout (standard output). This is not usually wished, but if you want to, you call nextflow with -process.echo and you will be able to read what's being written to stdout 😉

[mribeirodantas]

It allows your pipeline to write to stdout (standard output). This is not usually wished, but if you want to, you call nextflow with -process.echo and you will be able to read what's being written to stdout 😉

Oh, and yes, you can write code in whatever interpretable language you want to in the block. Python, R, whatever 😃

[asmlgkj]

thanks a lot. I just want to test if the script work as expected.

nextflow.enable.dsl=2
process myTask {
    input:
    val str

    script:
    '''
    #!/usr/bin/sh
    echo User ${str}
    '''
}

workflow {
    myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

N E X T F L O W ~ version 21.10.6 Launching `n4.nf` [exotic_lovelace] - revision: 91a787e2d3 executor > local (3) [87/59ba05] process > myTask (1) [100%] 3 of 3 ✔

why still not sdout?

[mribeirodantas]

If this code you shared is in a file named hello.nf, you have to run the following command: nextflow run hello.nf -process.echo. Then you should see the messages. Let me know if you don't.

PS: DSL2 is currently the default DSL. You don't have to specify it at the top of your file anymore.

[asmlgkj]

thanks a lot.
so any stout must use -process.echo?
because I think the shell command could output as well

[mribeirodantas]

You usually want to write to log files, not to the standard output. Imagine you have numerous processes, subworkflows, and so on. Your standard output will already be messy. It's not a good place to write to, usually.

But if you really want to, run with -process.echo 😄

[asmlgkj]

thanks a lot. how can this work?

nextflow.enable.dsl=2
process myTask {
    input:
    val str

    python:
    '''
    print(!${str})
    '''
}

workflow {
    myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

[mribeirodantas]

You can make this work by saving the code below to hello.nf and then running nextflow run hello.nf -process.echo

nextflow.enable.dsl=2
process myTask {
    input:
    val str

    shell:
    '''
    #!/usr/bin/env python
    print '!{str}'
    '''
}

workflow {
    myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

PS: Don't forget to accept an answer, so that this discussion becomes "answered" 😉.

[asmlgkj]

@mribeirodantas thanks a lot for your kind help.
I will mark that. because I evev used is issue almost.

Scripts à la carte
The process script is interpreted by Nextflow as a Bash script by default, but you are not limited to it.

You can use your favourite scripting language (e.g. Perl, Python, Ruby, R, etc), or even mix them in
but following command still does not work

nextflow.enable.dsl=2
process myTask {
    input:
    val str

    script:
    '''
    #!/usr/bin/env python
    print('!{str}')
    '''
}

workflow {
    myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

output is

and why there is extra \n

and my default python is python3, even the script is following, it does not say error (in python3, print need parentheses) ``` nextflow.enable.dsl=2 process myTask { input: val str

script:
'''
#!/usr/bin/env python
print '!{str}'
'''

}

workflow {
myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

[mribeirodantas]

You marked your message as the answer 😅

If you want to use Python3 as interpreter, you have to make the script be run by Python 3, not 2 😄
I tested the code below and it works:

nextflow.enable.dsl=2
process myTask {
    input:
    val str

    shell:
    '''
    #!/usr/bin/env python3
    print('!{str}')
    '''
}

workflow {
    myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

As for the newline, you have three strings in a channel. The process (that prints a string to the standard output) gets an element from the channel (string) and does what it is supposed to do; In this case, print. Then, next element. Print. Then, next element. Print. That's why you have the three strings in three separate lines.

[asmlgkj]

thanks a lot. do I mark right now?

I am talking about this two red new line

how can keyword script work? my example write the code in script block, yours in shell block? Can it only be used in shell block?

[mribeirodantas]

Now, it's unanswered again haha. Based on your original question, I assume this is an appropriate answer.

As for these empty lines, it's not related to Nextflow. It's the way the print function works in Python 3. If you don't want that, you have to pass an extra argument. The new code for the nextflow pipeline should then be:

nextflow.enable.dsl=2
process myTask {
    input:
    val str

    shell:
    '''
    #!/usr/bin/env python3
    print('!{str}', end = '')
    '''
}

workflow {
    myTask(Channel.of('Hello', 'Hola', 'Bonjour'))
}

Pay attention to the new parameter I added to the Python print function. Save it as hello.nf and then call it with: nextflow run test.nf -process.echo. For the differences between shell, script, and exec, you can find when to use each here. A good piece of advice is to create a new question whenever your doubts stray too far from the original question. We discussed many different issues here and this can prove hard to find for other users who can have similar questions 😅

Also, it may be better to join the Nextflow Slack Community Chat to have more extended discussions such as the one we had here. Have a nice day 😄