Running Dorado on Nextflow #4419

setshabaTaukobong · 2023-10-16T18:38:09Z

setshabaTaukobong
Oct 16, 2023

Hi there, I am trying to do basecalling using dorado on nextflow (on a HPC) however I keep getting no outputs. when I check the log file, this is what I get:
Oct-10 10:38:17.203 [main] DEBUG nextflow.cli.Launcher - $> nextflow run nextflow.log Oct-10 10:38:17.283 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 22.04.1 Oct-10 10:38:17.366 [main] DEBUG nextflow.scm.AssetManager - Listing projects in folder: /home/staukobong/.nextflow/assets Oct-10 10:38:17.397 [main] INFO nextflow.cli.CmdRun - Pulling nextflow-io/nextflow.log ... Oct-10 10:38:17.398 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials -:-] -> https://api.github.com/repos/nextflow-io/nextflow.log/contents/nextflow.config Oct-10 10:38:18.463 [main] DEBUG nextflow.scm.RepositoryProvider - Response status: 404 -- {"message":"Not Found","documentation_url":"https://docs.github.com/rest/repos/contents#get-repository-content"} Oct-10 10:38:18.471 [main] WARN nextflow.scm.AssetManager - Cannot read project manifest -- Cause: Remote resource not found: https://api.github.com/repos/nextflow-io/nextflow.log/contents/nextflow.config nextflow.exception.AbortOperationException: Remote resource not found: https://api.github.com/repos/nextflow-io/nextflow.log/contents/nextflow.config at nextflow.scm.RepositoryProvider.checkResponse(RepositoryProvider.groovy:189) at nextflow.scm.RepositoryProvider.invoke(RepositoryProvider.groovy:138) at nextflow.scm.RepositoryProvider.memoizedMethodPriv$invokeAndParseResponseString(RepositoryProvider.groovy:220) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1268) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035) at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1029) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1012) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:101) at nextflow.scm.RepositoryProvider$_closure2.doCall(RepositoryProvider.groovy) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035) at groovy.lang.Closure.call(Closure.java:412) at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137) at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137) at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113) at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136) at groovy.lang.Closure.call(Closure.java:428) at nextflow.scm.RepositoryProvider.invokeAndParseResponse(RepositoryProvider.groovy) at nextflow.scm.GithubRepositoryProvider.readBytes(GithubRepositoryProvider.groovy:100) at nextflow.scm.RepositoryProvider.readText(RepositoryProvider.groovy:234) at nextflow.scm.AssetManager.getManifest0(AssetManager.groovy:457) at nextflow.scm.AssetManager.memoizedMethodPriv$getManifest(AssetManager.groovy:450) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1268) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035) at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1029) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1012) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:101) at nextflow.scm.AssetManager$_closure2.doCall(AssetManager.groovy) at nextflow.scm.AssetManager$_closure2.doCall(AssetManager.groovy) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035) at groovy.lang.Closure.call(Closure.java:412) at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137) at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137) at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113) at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136) at groovy.lang.Closure.call(Closure.java:406) at nextflow.scm.AssetManager.getManifest(AssetManager.groovy) at nextflow.scm.AssetManager.getMainScriptName(AssetManager.groovy:433) at nextflow.scm.AssetManager.checkValidRemoteRepo(AssetManager.groovy:391) at nextflow.scm.AssetManager.download(AssetManager.groovy:600) at nextflow.scm.AssetManager.download(AssetManager.groovy) at nextflow.cli.CmdRun.getScriptFile0(CmdRun.groovy:477) at nextflow.cli.CmdRun.getScriptFile(CmdRun.groovy:410) at nextflow.cli.CmdRun.run(CmdRun.groovy:287) at nextflow.cli.Launcher.run(Launcher.groovy:480) at nextflow.cli.Launcher.main(Launcher.groovy:639) Oct-10 10:38:18.479 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials -:-] -> https://api.github.com/repos/nextflow-io/nextflow.log/contents/main.nf Oct-10 10:38:18.701 [main] DEBUG nextflow.scm.RepositoryProvider - Response status: 404 -- {"message":"Not Found","documentation_url":"https://docs.github.com/rest/repos/contents#get-repository-content"} Oct-10 10:38:18.702 [main] DEBUG nextflow.cli.Launcher - Operation aborted nextflow.exception.AbortOperationException: Remote resource not found: https://api.github.com/repos/nextflow-io/nextflow.log/contents/main.nf at nextflow.scm.RepositoryProvider.checkResponse(RepositoryProvider.groovy:189) at nextflow.scm.RepositoryProvider.invoke(RepositoryProvider.groovy:138) at nextflow.scm.RepositoryProvider.memoizedMethodPriv$invokeAndParseResponseString(RepositoryProvider.groovy:220) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1268) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035) at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1029) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1012) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:101) at nextflow.scm.RepositoryProvider$_closure2.doCall(RepositoryProvider.groovy) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035) at groovy.lang.Closure.call(Closure.java:412) at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137) at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137) at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113) at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136) at groovy.lang.Closure.call(Closure.java:428) at nextflow.scm.RepositoryProvider.invokeAndParseResponse(RepositoryProvider.groovy) at nextflow.scm.GithubRepositoryProvider.readBytes(GithubRepositoryProvider.groovy:100) at nextflow.scm.RepositoryProvider.validateFor(RepositoryProvider.groovy:248) at nextflow.scm.AssetManager.checkValidRemoteRepo(AssetManager.groovy:392) at nextflow.scm.AssetManager.download(AssetManager.groovy:600) at nextflow.scm.AssetManager.download(AssetManager.groovy) at nextflow.cli.CmdRun.getScriptFile0(CmdRun.groovy:477) at nextflow.cli.CmdRun.getScriptFile(CmdRun.groovy:410) at nextflow.cli.CmdRun.run(CmdRun.groovy:287) at nextflow.cli.Launcher.run(Launcher.groovy:480) at nextflow.cli.Launcher.main(Launcher.groovy:639)

I used the below script:
`pods_ch = Channel.fromPath('home/staukobong/*.pod5')

process BLASTSEQ {

debug true

input:
tuple val(sample_id), path(sample_id_paths)

output:
tuple val(sample_id), path("${sample_id}.bam")

script:
"""
dorado basecaller [email protected] $sample_id_paths > ${sample_id}.bam
"""

}
workflow {
bam_ch = BLASTSEQ(pods_ch)
bam_ch.view()
}`

Could I please get some assistance if anyone has an idea what may be the issue.

pditommaso · 2023-10-16T18:56:09Z

pditommaso
Oct 16, 2023
Maintainer

First step, update your installation of nextflow. Is more than one year old

0 replies

setshabaTaukobong · 2023-10-17T07:16:57Z

setshabaTaukobong
Oct 17, 2023
Author

HI there, thank you for the prompt response.
I updated nextflow to the latest version (23.10.0) however I am still getting the same issue.

0 replies

pditommaso · 2023-10-17T07:38:16Z

pditommaso
Oct 17, 2023
Maintainer

Can you please upload the .nextflow.log file as an attachment ?

0 replies

setshabaTaukobong · 2023-10-17T07:51:09Z

setshabaTaukobong
Oct 17, 2023
Author

I've attached the .nextflow.log file (nextflow.log).
I have also attached the following screenshots:
nextflow script (test-nf), the script I use to submit to the job. schedular (job-sh) and a nextflow config file (nextflow-config)
nextflow.log

<img width="1470" alt="nextflow-config" src="https://github.com/nextflow-io/nextflow/assets/84866754/bbd018e7-244b-4c43-84bf-9346b4ef4bd3"

0 replies

pditommaso · 2023-10-17T08:16:15Z

pditommaso
Oct 17, 2023
Maintainer

Apparently, you are running nextflow run nextflow.log which makes no sense. The correct syntax is nextflow run <script file or project repo>

3 replies

setshabaTaukobong Oct 17, 2023
Author

Not sure what you mean but if you take a look at the job-sh screenshot, I am running nextflow run test.nf

setshabaTaukobong Oct 17, 2023
Author

So what I do is that I run qsub job.sh (which is a script that loads all the modules and then runs test.nf).

colindaven Oct 20, 2023

Dorado is working fine for me right now on a cluster.

You want to look up and setup your executor (cluster) in your nextflow.config

Then run

nextflow run main.nf -c nextflow.config

Nextflow will handle job submission for you

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Running Dorado on Nextflow #4419

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 5 comments 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Running Dorado on Nextflow #4419

Uh oh!

Uh oh!

setshabaTaukobong Oct 16, 2023

Replies: 5 comments · 3 replies

Uh oh!

pditommaso Oct 16, 2023 Maintainer

Uh oh!

setshabaTaukobong Oct 17, 2023 Author

Uh oh!

pditommaso Oct 17, 2023 Maintainer

Uh oh!

setshabaTaukobong Oct 17, 2023 Author

Uh oh!

pditommaso Oct 17, 2023 Maintainer

Uh oh!

setshabaTaukobong Oct 17, 2023 Author

Uh oh!

setshabaTaukobong Oct 17, 2023 Author

Uh oh!

colindaven Oct 20, 2023

setshabaTaukobong
Oct 16, 2023

Replies: 5 comments 3 replies

pditommaso
Oct 16, 2023
Maintainer

setshabaTaukobong
Oct 17, 2023
Author

pditommaso
Oct 17, 2023
Maintainer

setshabaTaukobong
Oct 17, 2023
Author

pditommaso
Oct 17, 2023
Maintainer

setshabaTaukobong Oct 17, 2023
Author

setshabaTaukobong Oct 17, 2023
Author