Workflow Composition Isolation

[qscacheri]

Hello! I'm wondering if anyone could help me with a problem I'm having with dsl2 and workflow composition. I am trying to run two workflows in parallel from a main workflow by doing something like foo(channel.of("a", "b"). However, when the workflows run, I'm getting a mix of a and b. I've included a minimal script to show what I'm talking about. I would have assumed that the script would output:

[a-first-time-second-time, a-first-time]
[b-first-time-second-time, b-first-time]

but it actually outputs:

[b-first-time-second-time, b-first-time]
[b-first-time-second-time, a-first-time]
[a-first-time-second-time, b-first-time]
[a-first-time-second-time, a-first-time]

I'm new to nextflow so this could totally be a misunderstanding but any help would be greatly appreciated.

process bar {
    input:
        val v
    output:
        env out
    script:
    """
    out="$v-first-time"
    """
}

process baz {
    input: 
        val v
    output:
        env out
    script:
    """
    out="$v-second-time"
    """
}

workflow foo {
    take: 
    testValue
    main: 
    bar(testValue)    
    baz(bar.out)  
    baz.out.combine(bar.out).view()
}

workflow {
    foo(channel.of('a','b')) 
}

[bentsherman]

Hi @qscacheri , answering for posterity since no one ever answered you. This is a common pitfall when learning how to write Nextflow pipelines. Basically, processes do not necessarily emit outputs in the same order as the inputs, so in this case bar.out and baz.out may not have the same order. You should always add some metadata (e.g. id) to each item so that you can recombine outputs later on.

Additionally, the combine operator does an outer product whereas it looks like you wanted an inner product. For that you should use join.