The max number of supported channels is 255 #2414

kevyin · 2021-10-27T09:50:42Z

kevyin
Oct 27, 2021

Bug report

Expected behavior and actual behavior

In DSL2 there is a limit to the number of channels/references to channels in a *.nf script

Expected behavior is that there is no such limit

Steps to reproduce the problem

I've attached a toy example in the comment below
#2411 (comment)
Also attached: https://github.com/nextflow-io/nextflow/files/7426077/mytest.nf.zip

touch touchfile
nextflow run test.nf --myfile touchfile

Program output

N E X T F L O W  ~  version 21.04.3
Launching `test.nf` [golden_montalcini] - revision: c44de35a33
General error during instruction selection: The max number of supported arguments is 255, but found 262

java.lang.IllegalArgumentException: The max number of supported arguments is 255, but found 262
        at org.codehaus.groovy.classgen.asm.CallSiteWriter.getCreateArraySignature(CallSiteWriter.java:74)
        at org.codehaus.groovy.classgen.asm.CallSiteWriter.makeCallSite(CallSiteWriter.java:340)
        at org.codehaus.groovy.classgen.asm.InvocationWriter.writeNormalConstructorCall(InvocationWriter.java:562)
        at org.codehaus.groovy.classgen.asm.InvocationWriter.writeInvokeConstructor(InvocationWriter.java:568)
        at org.codehaus.groovy.classgen.AsmClassGenerator.visitConstructorCallExpression(AsmClassGenerator.java:862)
        at org.codehaus.groovy.ast.expr.ConstructorCallExpression.visit(ConstructorCallExpression.java:44)
        at org.codehaus.groovy.classgen.asm.StatementWriter.writeReturn(StatementWriter.java:613)
        at org.codehaus.groovy.classgen.asm.OptimizingStatementWriter.writeReturn(OptimizingStatementWriter.java:322)
        at org.codehaus.groovy.classgen.AsmClassGenerator.visitReturnStatement(AsmClassGenerator.java:681)
        at org.codehaus.groovy.ast.stmt.ReturnStatement.visit(ReturnStatement.java:73)
        at org.codehaus.groovy.classgen.asm.StatementWriter.writeBlockStatement(StatementWriter.java:94)
        at org.codehaus.groovy.classgen.asm.OptimizingStatementWriter.writeBlockStatement(OptimizingStatementWriter.java:169)
        at org.codehaus.groovy.classgen.AsmClassGenerator.visitBlockStatement(AsmClassGenerator.java:617)
        at org.codehaus.groovy.ast.stmt.BlockStatement.visit(BlockStatement.java:69)
        at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClassCodeContainer(ClassCodeVisitorSupport.java:138)
        at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitConstructorOrMethod(ClassCodeVisitorSupport.java:111)
        at org.codehaus.groovy.classgen.AsmClassGenerator.visitStdMethod(AsmClassGenerator.java:461)
        at org.codehaus.groovy.classgen.AsmClassGenerator.visitConstructorOrMethod(AsmClassGenerator.java:408)
        at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitMethod(ClassCodeVisitorSupport.java:106)
        at org.codehaus.groovy.classgen.AsmClassGenerator.visitMethod(AsmClassGenerator.java:556)
        at org.codehaus.groovy.ast.ClassNode.visitMethods(ClassNode.java:1099)
        at org.codehaus.groovy.ast.ClassNode.visitContents(ClassNode.java:1092)
        at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClass(ClassCodeVisitorSupport.java:52)
        at org.codehaus.groovy.classgen.AsmClassGenerator.visitClass(AsmClassGenerator.java:271)
        at org.codehaus.groovy.control.CompilationUnit$3.call(CompilationUnit.java:797)
        at org.codehaus.groovy.control.CompilationUnit$3.call(CompilationUnit.java:813)
        at org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:942)
        at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:671)
        at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:635)
        at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:389)
        at groovy.lang.GroovyClassLoader.lambda$parseClass$3(GroovyClassLoader.java:332)
        at org.codehaus.groovy.runtime.memoize.StampedCommonCache.compute(StampedCommonCache.java:163)
        at org.codehaus.groovy.runtime.memoize.StampedCommonCache.getAndPut(StampedCommonCache.java:154)
        at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:330)
        at groovy.lang.GroovyShell.parseClass(GroovyShell.java:526)
        at groovy.lang.GroovyShell.parse(GroovyShell.java:538)
        at groovy.lang.GroovyShell.parse(GroovyShell.java:570)
        at nextflow.script.ScriptParser.parse(ScriptParser.groovy:172)
        at nextflow.script.ScriptParser.parse(ScriptParser.groovy:191)
        at nextflow.script.ScriptParser.parse(ScriptParser.groovy:196)
        at nextflow.script.ScriptRunner.parseScript(ScriptRunner.groovy:198)
        at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:118)
        at nextflow.cli.CmdRun.run(CmdRun.groovy:302)
        at nextflow.cli.Launcher.run(Launcher.groovy:475)
        at nextflow.cli.Launcher.main(Launcher.groovy:657)

1 error

Environment

Nextflow version: 21.04.3
Java version: openjdk version "1.8.0_292"
OpenJDK Runtime Environment (build 1.8.0_292-8u292-b10-0ubuntu1~20.04-b10)
OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)
Operating system: Linux
Bash version: GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)

Additional context

nextflow.log

(Add any other context about the problem here)

kevyin · 2021-10-27T09:51:59Z

kevyin
Oct 27, 2021
Author

// test.nf

nextflow.enable.dsl=2

process test {
    input:
    file myfile

    """
    echo myfile

    """

}

workflow {
    inputfile = Channel.from(file(params.myfile))
    test(inputfile)

    inputfile_tmp1 = Channel.from(file(params.myfile))
    inputfile_tmp2 = Channel.from(file(params.myfile))
    inputfile_tmp3 = Channel.from(file(params.myfile))
    inputfile_tmp4 = Channel.from(file(params.myfile))
    inputfile_tmp5 = Channel.from(file(params.myfile))
    inputfile_tmp6 = Channel.from(file(params.myfile))
    inputfile_tmp7 = Channel.from(file(params.myfile))
    inputfile_tmp8 = Channel.from(file(params.myfile))
    inputfile_tmp9 = Channel.from(file(params.myfile))
    inputfile_tmp10 = Channel.from(file(params.myfile))
    inputfile_tmp11 = Channel.from(file(params.myfile))
    inputfile_tmp12 = Channel.from(file(params.myfile))
    inputfile_tmp13 = Channel.from(file(params.myfile))
    inputfile_tmp14 = Channel.from(file(params.myfile))
    inputfile_tmp15 = Channel.from(file(params.myfile))
    inputfile_tmp16 = Channel.from(file(params.myfile))
    inputfile_tmp17 = Channel.from(file(params.myfile))
    inputfile_tmp18 = Channel.from(file(params.myfile))
    inputfile_tmp19 = Channel.from(file(params.myfile))
    inputfile_tmp20 = Channel.from(file(params.myfile))
    inputfile_tmp21 = Channel.from(file(params.myfile))
    inputfile_tmp22 = Channel.from(file(params.myfile))
    inputfile_tmp23 = Channel.from(file(params.myfile))
    inputfile_tmp24 = Channel.from(file(params.myfile))
    inputfile_tmp25 = Channel.from(file(params.myfile))
    inputfile_tmp26 = Channel.from(file(params.myfile))
    inputfile_tmp27 = Channel.from(file(params.myfile))
    inputfile_tmp28 = Channel.from(file(params.myfile))
    inputfile_tmp29 = Channel.from(file(params.myfile))
    inputfile_tmp30 = Channel.from(file(params.myfile))
    inputfile_tmp31 = Channel.from(file(params.myfile))
    inputfile_tmp32 = Channel.from(file(params.myfile))
    inputfile_tmp33 = Channel.from(file(params.myfile))
    inputfile_tmp34 = Channel.from(file(params.myfile))
    inputfile_tmp35 = Channel.from(file(params.myfile))
    inputfile_tmp36 = Channel.from(file(params.myfile))
    inputfile_tmp37 = Channel.from(file(params.myfile))
    inputfile_tmp38 = Channel.from(file(params.myfile))
    inputfile_tmp39 = Channel.from(file(params.myfile))
    inputfile_tmp40 = Channel.from(file(params.myfile))
    inputfile_tmp41 = Channel.from(file(params.myfile))
    inputfile_tmp42 = Channel.from(file(params.myfile))
    inputfile_tmp43 = Channel.from(file(params.myfile))
    inputfile_tmp44 = Channel.from(file(params.myfile))
    inputfile_tmp45 = Channel.from(file(params.myfile))
    inputfile_tmp46 = Channel.from(file(params.myfile))
    inputfile_tmp47 = Channel.from(file(params.myfile))
    inputfile_tmp48 = Channel.from(file(params.myfile))
    inputfile_tmp49 = Channel.from(file(params.myfile))
    inputfile_tmp50 = Channel.from(file(params.myfile))
    inputfile_tmp51 = Channel.from(file(params.myfile))
    inputfile_tmp52 = Channel.from(file(params.myfile))
    inputfile_tmp53 = Channel.from(file(params.myfile))
    inputfile_tmp54 = Channel.from(file(params.myfile))
    inputfile_tmp55 = Channel.from(file(params.myfile))
    inputfile_tmp56 = Channel.from(file(params.myfile))
    inputfile_tmp57 = Channel.from(file(params.myfile))
    inputfile_tmp58 = Channel.from(file(params.myfile))
    inputfile_tmp59 = Channel.from(file(params.myfile))
    inputfile_tmp60 = Channel.from(file(params.myfile))
    inputfile_tmp61 = Channel.from(file(params.myfile))
    inputfile_tmp62 = Channel.from(file(params.myfile))
    inputfile_tmp63 = Channel.from(file(params.myfile))
    inputfile_tmp64 = Channel.from(file(params.myfile))
    inputfile_tmp65 = Channel.from(file(params.myfile))
    inputfile_tmp66 = Channel.from(file(params.myfile))
    inputfile_tmp67 = Channel.from(file(params.myfile))
    inputfile_tmp68 = Channel.from(file(params.myfile))
    inputfile_tmp69 = Channel.from(file(params.myfile))
    inputfile_tmp70 = Channel.from(file(params.myfile))
    inputfile_tmp71 = Channel.from(file(params.myfile))
    inputfile_tmp72 = Channel.from(file(params.myfile))
    inputfile_tmp73 = Channel.from(file(params.myfile))
    inputfile_tmp74 = Channel.from(file(params.myfile))
    inputfile_tmp75 = Channel.from(file(params.myfile))
    inputfile_tmp76 = Channel.from(file(params.myfile))
    inputfile_tmp77 = Channel.from(file(params.myfile))
    inputfile_tmp78 = Channel.from(file(params.myfile))
    inputfile_tmp79 = Channel.from(file(params.myfile))
    inputfile_tmp80 = Channel.from(file(params.myfile))
    inputfile_tmp81 = Channel.from(file(params.myfile))
    inputfile_tmp82 = Channel.from(file(params.myfile))
    inputfile_tmp83 = Channel.from(file(params.myfile))
    inputfile_tmp84 = Channel.from(file(params.myfile))
    inputfile_tmp85 = Channel.from(file(params.myfile))
    inputfile_tmp86 = Channel.from(file(params.myfile))
    inputfile_tmp87 = Channel.from(file(params.myfile))
    inputfile_tmp88 = Channel.from(file(params.myfile))
    inputfile_tmp89 = Channel.from(file(params.myfile))
    inputfile_tmp90 = Channel.from(file(params.myfile))
    inputfile_tmp91 = Channel.from(file(params.myfile))
    inputfile_tmp92 = Channel.from(file(params.myfile))
    inputfile_tmp93 = Channel.from(file(params.myfile))
    inputfile_tmp94 = Channel.from(file(params.myfile))
    inputfile_tmp95 = Channel.from(file(params.myfile))
    inputfile_tmp96 = Channel.from(file(params.myfile))
    inputfile_tmp97 = Channel.from(file(params.myfile))
    inputfile_tmp98 = Channel.from(file(params.myfile))
    inputfile_tmp99 = Channel.from(file(params.myfile))
    inputfile_tmp100 = Channel.from(file(params.myfile))
    inputfile_tmp101 = Channel.from(file(params.myfile))
    inputfile_tmp102 = Channel.from(file(params.myfile))
    inputfile_tmp103 = Channel.from(file(params.myfile))
    inputfile_tmp104 = Channel.from(file(params.myfile))
    inputfile_tmp105 = Channel.from(file(params.myfile))
    inputfile_tmp106 = Channel.from(file(params.myfile))
    inputfile_tmp107 = Channel.from(file(params.myfile))
    inputfile_tmp108 = Channel.from(file(params.myfile))
    inputfile_tmp109 = Channel.from(file(params.myfile))
    inputfile_tmp110 = Channel.from(file(params.myfile))
    inputfile_tmp111 = Channel.from(file(params.myfile))
    inputfile_tmp112 = Channel.from(file(params.myfile))
    inputfile_tmp113 = Channel.from(file(params.myfile))
    inputfile_tmp114 = Channel.from(file(params.myfile))
    inputfile_tmp115 = Channel.from(file(params.myfile))
    inputfile_tmp116 = Channel.from(file(params.myfile))
    inputfile_tmp117 = Channel.from(file(params.myfile))
    inputfile_tmp118 = Channel.from(file(params.myfile))
    inputfile_tmp119 = Channel.from(file(params.myfile))
    inputfile_tmp120 = Channel.from(file(params.myfile))
    inputfile_tmp121 = Channel.from(file(params.myfile))
    inputfile_tmp122 = Channel.from(file(params.myfile))
    inputfile_tmp123 = Channel.from(file(params.myfile))
    inputfile_tmp124 = Channel.from(file(params.myfile))
    inputfile_tmp125 = Channel.from(file(params.myfile))
    inputfile_tmp126 = Channel.from(file(params.myfile))
    inputfile_tmp127 = Channel.from(file(params.myfile))
    inputfile_tmp128 = Channel.from(file(params.myfile))
    inputfile_tmp129 = Channel.from(file(params.myfile))
    inputfile_tmp130 = Channel.from(file(params.myfile))
    inputfile_tmp131 = Channel.from(file(params.myfile))
    inputfile_tmp132 = Channel.from(file(params.myfile))
    inputfile_tmp133 = Channel.from(file(params.myfile))
    inputfile_tmp134 = Channel.from(file(params.myfile))
    inputfile_tmp135 = Channel.from(file(params.myfile))
    inputfile_tmp136 = Channel.from(file(params.myfile))
    inputfile_tmp137 = Channel.from(file(params.myfile))
    inputfile_tmp138 = Channel.from(file(params.myfile))
    inputfile_tmp139 = Channel.from(file(params.myfile))
    inputfile_tmp140 = Channel.from(file(params.myfile))
    inputfile_tmp141 = Channel.from(file(params.myfile))
    inputfile_tmp142 = Channel.from(file(params.myfile))
    inputfile_tmp143 = Channel.from(file(params.myfile))
    inputfile_tmp144 = Channel.from(file(params.myfile))
    inputfile_tmp145 = Channel.from(file(params.myfile))
    inputfile_tmp146 = Channel.from(file(params.myfile))
    inputfile_tmp147 = Channel.from(file(params.myfile))
    inputfile_tmp148 = Channel.from(file(params.myfile))
    inputfile_tmp149 = Channel.from(file(params.myfile))
    inputfile_tmp150 = Channel.from(file(params.myfile))
    inputfile_tmp151 = Channel.from(file(params.myfile))
    inputfile_tmp152 = Channel.from(file(params.myfile))
    inputfile_tmp153 = Channel.from(file(params.myfile))
    inputfile_tmp154 = Channel.from(file(params.myfile))
    inputfile_tmp155 = Channel.from(file(params.myfile))
    inputfile_tmp156 = Channel.from(file(params.myfile))
    inputfile_tmp157 = Channel.from(file(params.myfile))
    inputfile_tmp158 = Channel.from(file(params.myfile))
    inputfile_tmp159 = Channel.from(file(params.myfile))
    inputfile_tmp160 = Channel.from(file(params.myfile))
    inputfile_tmp161 = Channel.from(file(params.myfile))
    inputfile_tmp162 = Channel.from(file(params.myfile))
    inputfile_tmp163 = Channel.from(file(params.myfile))
    inputfile_tmp164 = Channel.from(file(params.myfile))
    inputfile_tmp165 = Channel.from(file(params.myfile))
    inputfile_tmp166 = Channel.from(file(params.myfile))
    inputfile_tmp167 = Channel.from(file(params.myfile))
    inputfile_tmp168 = Channel.from(file(params.myfile))
    inputfile_tmp169 = Channel.from(file(params.myfile))
    inputfile_tmp170 = Channel.from(file(params.myfile))
    inputfile_tmp171 = Channel.from(file(params.myfile))
    inputfile_tmp172 = Channel.from(file(params.myfile))
    inputfile_tmp173 = Channel.from(file(params.myfile))
    inputfile_tmp174 = Channel.from(file(params.myfile))
    inputfile_tmp175 = Channel.from(file(params.myfile))
    inputfile_tmp176 = Channel.from(file(params.myfile))
    inputfile_tmp177 = Channel.from(file(params.myfile))
    inputfile_tmp178 = Channel.from(file(params.myfile))
    inputfile_tmp179 = Channel.from(file(params.myfile))
    inputfile_tmp180 = Channel.from(file(params.myfile))
    inputfile_tmp181 = Channel.from(file(params.myfile))
    inputfile_tmp182 = Channel.from(file(params.myfile))
    inputfile_tmp183 = Channel.from(file(params.myfile))
    inputfile_tmp184 = Channel.from(file(params.myfile))
    inputfile_tmp185 = Channel.from(file(params.myfile))
    inputfile_tmp186 = Channel.from(file(params.myfile))
    inputfile_tmp187 = Channel.from(file(params.myfile))
    inputfile_tmp188 = Channel.from(file(params.myfile))
    inputfile_tmp189 = Channel.from(file(params.myfile))
    inputfile_tmp190 = Channel.from(file(params.myfile))
    inputfile_tmp191 = Channel.from(file(params.myfile))
    inputfile_tmp192 = Channel.from(file(params.myfile))
    inputfile_tmp193 = Channel.from(file(params.myfile))
    inputfile_tmp194 = Channel.from(file(params.myfile))
    inputfile_tmp195 = Channel.from(file(params.myfile))
    inputfile_tmp196 = Channel.from(file(params.myfile))
    inputfile_tmp197 = Channel.from(file(params.myfile))
    inputfile_tmp198 = Channel.from(file(params.myfile))
    inputfile_tmp199 = Channel.from(file(params.myfile))
    inputfile_tmp200 = Channel.from(file(params.myfile))
    inputfile_tmp201 = Channel.from(file(params.myfile))
    inputfile_tmp202 = Channel.from(file(params.myfile))
    inputfile_tmp203 = Channel.from(file(params.myfile))
    inputfile_tmp204 = Channel.from(file(params.myfile))
    inputfile_tmp205 = Channel.from(file(params.myfile))
    inputfile_tmp206 = Channel.from(file(params.myfile))
    inputfile_tmp207 = Channel.from(file(params.myfile))
    inputfile_tmp208 = Channel.from(file(params.myfile))
    inputfile_tmp209 = Channel.from(file(params.myfile))
    inputfile_tmp210 = Channel.from(file(params.myfile))
    inputfile_tmp211 = Channel.from(file(params.myfile))
    inputfile_tmp212 = Channel.from(file(params.myfile))
    inputfile_tmp213 = Channel.from(file(params.myfile))
    inputfile_tmp214 = Channel.from(file(params.myfile))
    inputfile_tmp215 = Channel.from(file(params.myfile))
    inputfile_tmp216 = Channel.from(file(params.myfile))
    inputfile_tmp217 = Channel.from(file(params.myfile))
    inputfile_tmp218 = Channel.from(file(params.myfile))
    inputfile_tmp219 = Channel.from(file(params.myfile))
    inputfile_tmp220 = Channel.from(file(params.myfile))
    inputfile_tmp221 = Channel.from(file(params.myfile))
    inputfile_tmp222 = Channel.from(file(params.myfile))
    inputfile_tmp223 = Channel.from(file(params.myfile))
    inputfile_tmp224 = Channel.from(file(params.myfile))
    inputfile_tmp225 = Channel.from(file(params.myfile))
    inputfile_tmp226 = Channel.from(file(params.myfile))
    inputfile_tmp227 = Channel.from(file(params.myfile))
    inputfile_tmp228 = Channel.from(file(params.myfile))
    inputfile_tmp229 = Channel.from(file(params.myfile))
    inputfile_tmp230 = Channel.from(file(params.myfile))
    inputfile_tmp231 = Channel.from(file(params.myfile))
    inputfile_tmp232 = Channel.from(file(params.myfile))
    inputfile_tmp233 = Channel.from(file(params.myfile))
    inputfile_tmp234 = Channel.from(file(params.myfile))
    inputfile_tmp235 = Channel.from(file(params.myfile))
    inputfile_tmp236 = Channel.from(file(params.myfile))
    inputfile_tmp237 = Channel.from(file(params.myfile))
    inputfile_tmp238 = Channel.from(file(params.myfile))
    inputfile_tmp239 = Channel.from(file(params.myfile))
    inputfile_tmp240 = Channel.from(file(params.myfile))
    inputfile_tmp241 = Channel.from(file(params.myfile))
    inputfile_tmp242 = Channel.from(file(params.myfile))
    inputfile_tmp243 = Channel.from(file(params.myfile))
    inputfile_tmp244 = Channel.from(file(params.myfile))
// comment out some of these to make test pass
    inputfile_tmp245 = Channel.from(file(params.myfile))
    inputfile_tmp246 = Channel.from(file(params.myfile))
    inputfile_tmp247 = Channel.from(file(params.myfile))
    inputfile_tmp248 = Channel.from(file(params.myfile))
    inputfile_tmp249 = Channel.from(file(params.myfile))
    inputfile_tmp250 = Channel.from(file(params.myfile))
    inputfile_tmp251 = Channel.from(file(params.myfile))
    inputfile_tmp252 = Channel.from(file(params.myfile))
    inputfile_tmp253 = Channel.from(file(params.myfile))
    inputfile_tmp254 = Channel.from(file(params.myfile))
    inputfile_tmp255 = Channel.from(file(params.myfile))
    inputfile_tmp256 = Channel.from(file(params.myfile))
}

0 replies

abhi18av · 2021-10-27T10:09:45Z

abhi18av
Oct 27, 2021
Collaborator

Hi @kevyin ,

This is an interesting point - I'm not sure if this is a bug/feature request.

In any case, it could be resolved at design level since all of these params could essentially be wrapped into a tuple and then passed into the process.

And within the prcess you can use something as shown below

input:
path("*")

... to essentially use all of these together.

0 replies

kevyin · 2021-10-27T12:09:19Z

kevyin
Oct 27, 2021
Author

Hi @kevyin ,

This is an interesting point - I'm not sure if this is a bug/feature request.

In any case, it could be resolved at design level since all of these params could essentially be wrapped into a tuple and then passed into the process.

And within the prcess you can use something as shown below
input:
path("*")
... to essentially use all of these together.

Hi @abhi18av ,
Thanks for the reply,
Unfortunately the reality is I'm hitting this issue in a much more complicated and bigger workflow, not as clean as this toy example.

It's not just inputs, in the example, but all kinds of Channels that goes into the 255 maximum.
To illustrate, process output are also affected, and (I assume) results from channel operations as well

test(inputfile)
test_out0 = test.out[0]
test_out1 = test.out[1]

255 seems like some integer limit, 2^8 = 256
I wonder if it's possible to change to a larger type somewhere?

0 replies

pditommaso · 2021-10-27T12:19:43Z

pditommaso
Oct 27, 2021
Maintainer

I don't see how this can happen with the example provided?

touch touchfile
nextflow run test.nf --myfile touchfile

0 replies

kevyin · 2021-10-27T12:31:51Z

kevyin
Oct 27, 2021
Author

I don't see how this can happen with the example provided?
touch touchfile
nextflow run test.nf --myfile touchfile

@pditommaso
Sorry I wasnt clear,
I didnt know how to attach files to the issue so pasted test.nf in the comment above
#2411 (comment)

Here it is again attached
mytest.nf.zip

and the log
nextflow.log

0 replies

kevyin · 2021-10-27T14:05:59Z

kevyin
Oct 27, 2021
Author

I don't see how this can happen with the example provided?
touch touchfile
nextflow run test.nf --myfile touchfile

Hi @pditommaso @abhi18av
Apologies I just realized another confusion.

The actual error message says "arguments"
But its actually the max number of supported channels that's hitting a 255 limit

I've renamed the issue

4 replies

pditommaso Oct 27, 2021
Maintainer

Not sure what you are trying to achieve with this code.

kevyin Oct 27, 2021
Author

The actual code where I hit this issue is in a much bigger and complex pipeline.
This toy example is just trying to show that there is a limit to the number of Channel objects.

Attached is another test but each channel is created from output of test.out[0]

I've done more trial and error and at around 243 objects there's a different message

org.codehaus.groovy.runtime.ArrayUtil.createArray(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljav
Full error log:
243_channels_nextflow.log

Example with different channel operations
test_with_op.nf.tar.gz

After a certain point the error message changes, in test_with_op.nf, I've marked where the transition occurs

    inputfile_tmp240 = test.out[0]
    inputfile_tmp241 = test.out[0]
// comment out some of these to make test pass
    inputfile_tmp242 = test.out[0]
    inputfile_tmp243 = test.out[0] // past this point errors with  "org.codehaus.groovy.runtime.ArrayUtil.createArray(Ljava/lang/Object;Ljava/lang/Object;Lja....."
    inputfile_tmp244 = test.out[0]
    inputfile_tmp245 = test.out[0]
    inputfile_tmp246 = test.out[0]
    inputfile_tmp247 = test.out[0] // past this point errors with "The max number of supported arguments is 255, but found 255 ..."
//    inputfile_tmp248 = test.out[0]
//    inputfile_tmp249 = test.out[0]

kevyin Oct 27, 2021
Author

@pditommaso

Seems like this is related

https://github.com/groovy/groovy-core/blob/01309f9d4be34ddf93c4a9943b5a97843bff6181/src/main/org/codehaus/groovy/classgen/asm/CallSiteWriter.java#L57

Possibly a JVM limit?
https://stackoverflow.com/questions/30581531/maximum-number-of-parameters-in-java-method-declaration
https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.11

The number of method parameters is limited to 255 by the definition of a method descriptor (§4.3.3), where the limit includes one unit for this in the case of instance or interface method invocations.

Note that a method descriptor is defined in terms of a notion of method parameter length in which a parameter of type long or double contributes two units to the length, so parameters of these types further reduce the limit.

The number of dimensions in an array is limited to 255 by the size of the dimensions opcode of the multianewarray instruction and by the constraints imposed on the multianewarray, anewarray, and newarray instructions (§4.9.1, §4.9.2).

pditommaso Oct 28, 2021
Maintainer

Provided that having so many channels into a single workflow context signal a lack of modularization of your workflow, I think I've understood why it's happening. It should be possible to prevent it.

The max number of supported channels is 255 #2414

Uh oh!

Uh oh!

kevyin Oct 27, 2021

Bug report

Expected behavior and actual behavior

Steps to reproduce the problem

Program output

Environment

Additional context

Replies: 6 comments · 4 replies

Uh oh!

Uh oh!

kevyin Oct 27, 2021 Author

Uh oh!

abhi18av Oct 27, 2021 Collaborator

Uh oh!

Uh oh!

kevyin Oct 27, 2021 Author

Uh oh!

pditommaso Oct 27, 2021 Maintainer

Uh oh!

Uh oh!

kevyin Oct 27, 2021 Author

Uh oh!

kevyin Oct 27, 2021 Author

Uh oh!

pditommaso Oct 27, 2021 Maintainer

Uh oh!

kevyin Oct 27, 2021 Author

Uh oh!

Uh oh!

kevyin Oct 27, 2021 Author

Uh oh!

pditommaso Oct 28, 2021 Maintainer

kevyin
Oct 27, 2021

Replies: 6 comments 4 replies

kevyin
Oct 27, 2021
Author

abhi18av
Oct 27, 2021
Collaborator

kevyin
Oct 27, 2021
Author

pditommaso
Oct 27, 2021
Maintainer

kevyin
Oct 27, 2021
Author

kevyin
Oct 27, 2021
Author

pditommaso Oct 27, 2021
Maintainer

kevyin Oct 27, 2021
Author

kevyin Oct 27, 2021
Author

pditommaso Oct 28, 2021
Maintainer