Failed to run locally.

[shenwei356]

Hi, I'm trying to install and run BigSeqKit locally.

Here are the commands. There's an error of Permission denied, but it remains after changing ownership of the ignis directory.

# no error
sudo docker pull ignishpc/full

cd ~/Downloads
# no error
docker run --rm -v $(pwd):/target ignishpc/submitter ignis-export /target

# set current directory as job directory
export IGNIS_DFS_ID=$(pwd)
# set docker as scheduler
export IGNIS_SCHEDULER_TYPE=docker
# set where docker is available
export IGNIS_SCHEDULER_URL=/var/run/docker.sock

./ignis/bin/ignis-submit ignishpc/full bigseqkit
Mar 10, 2024 09:03:37 <ERROR> org.ignis.submitter.Submit.call(Submit.java:271): java.io.IOException: com.sun.jna.LastErrorException: [13] Permission denied
java.lang.RuntimeException: java.io.IOException: com.sun.jna.LastErrorException: [13] Permission denied
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:179) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:9) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.DefaultInvocationBuilder.post(DefaultInvocationBuilder.java:124) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.exec.CreateContainerCmdExec.execute(CreateContainerCmdExec.java:37) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.exec.CreateContainerCmdExec.execute(CreateContainerCmdExec.java:13) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.command.CreateContainerCmdImpl.exec(CreateContainerCmdImpl.java:609) ~[docker-java-core-3.2.11.jar:?]
        at org.ignis.scheduler.Docker.createDriverContainer(Docker.java:318) ~[ignis-scheduler-docker-1.0.jar:?]
        at org.ignis.submitter.Submit.call(Submit.java:186) [ignis-submitter-1.0.jar:?]
        at org.ignis.submitter.Submit.call(Submit.java:46) [ignis-submitter-1.0.jar:?]
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine.access$1300(CommandLine.java:145) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine.execute(CommandLine.java:2078) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at org.ignis.submitter.Submit.main(Submit.java:284) [ignis-submitter-1.0.jar:?]
Caused by: java.io.IOException: com.sun.jna.LastErrorException: [13] Permission denied
        at com.github.dockerjava.transport.DomainSocket.<init>(DomainSocket.java:63) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.LinuxDomainSocket.<init>(LinuxDomainSocket.java:41) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.DomainSocket.get(DomainSocket.java:138) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl$2.createSocket(ApacheDockerHttpClientImpl.java:137) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:125) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:409) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:164) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:174) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:135) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:172) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:93) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:128) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:178) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:67) ~[httpclient5-5.0.3.jar:5.0.3]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:175) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        ... 19 more
Caused by: com.sun.jna.LastErrorException: [13] Permission denied
        at com.github.dockerjava.transport.LinuxDomainSocket.connect(Native Method) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.LinuxDomainSocket.connect(LinuxDomainSocket.java:49) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.DomainSocket.open(DomainSocket.java:69) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.DomainSocket.<init>(DomainSocket.java:59) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.LinuxDomainSocket.<init>(LinuxDomainSocket.java:41) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.DomainSocket.get(DomainSocket.java:138) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl$2.createSocket(ApacheDockerHttpClientImpl.java:137) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:125) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:409) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:164) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:174) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:135) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:172) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:93) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:128) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:178) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:67) ~[httpclient5-5.0.3.jar:5.0.3]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:175) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        ... 19 more

[cesarpomar]

The Permission denied error is because your user doesn't have permission to access /var/run/docker.sock. Please check if you can run docker ps.

Another option is to run ignis-submit command with sudo.

Please let me know if you find any other errors

[shenwei356]

$ sudo docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

$ sudo ./ignis/bin/ignis-submit ignishpc/full bigseqkit
Mar 12, 2024 09:59:28 <ERROR> org.ignis.submitter.Submit.call(Submit.java:271): Key 'ignis.scheduler.type' value error,  not found
org.ignis.properties.IPropertyException: Key 'ignis.scheduler.type' value error,  not found
        at org.ignis.properties.IProperties.getProperty(IProperties.java:69) ~[ignis-properties-1.0.jar:?]
        at org.ignis.submitter.Submit.call(Submit.java:113) [ignis-submitter-1.0.jar:?]
        at org.ignis.submitter.Submit.call(Submit.java:46) [ignis-submitter-1.0.jar:?]
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine.access$1300(CommandLine.java:145) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine.execute(CommandLine.java:2078) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at org.ignis.submitter.Submit.main(Submit.java:284) [ignis-submitter-1.0.jar:?]

[cesarpomar]

You should use sudo -E to preserve the environment variables defined with the export command. The error indicates that ignis.scheduler.type created with export IGNIS_SCHEDULER_TYPE=docker does not exist.

[shenwei356]

Oh thanks, it finally worked. However a new error comes out. It reported a file not exist error. But the input file does exist.

$ ll /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa
-rw-r--r-- 1 shenwei shenwei 4.6M  9月25日 08:16 /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa

$ sudo -E sh ./ignis/bin/ignis-submit ignishpc/full 'bigseqkit seq /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa | bigseqkit stats'

...
panic: stat /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa: no such file or directory
...

details.

Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:46): Backend started
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:49): Loading environment variables
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:64): Loading configuration file
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:80): Loading scheduler
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:92): Checking scheduler docker
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:95): Scheduler docker /var/run/docker.sock ... OK
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:103): Getting Driver container info from scheduler
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:105): Driver container info found
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:125): Setting dynamic properties
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.services.IBackendServiceImpl.start(IBackendServiceImpl.java:48): Backend server started on port 4000
[INFO]  2024/03/12 14:14:35 ilog.go:37: ServerModule: go executor started
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.services.IBackendServiceImpl.startHealthServer(IBackendServiceImpl.java:117): Backend health server started
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.cluster.IClusterCreateHelper.create(IClusterCreateHelper.java:53): Cluster(0): Registering cluster with 1 containers
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.worker.IWorkerCreateHelper.create(IWorkerCreateHelper.java:57): Cluster(0), Worker(0): Registering worker with 1 executors with 1 cores
panic: stat /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa: no such file or directory

goroutine 1 [running]:
main.checkError(...)
    /tmp/src/bigseqkit-cli/helper.go:34
main.getFileList({0xc000344720, 0x3, 0x3}, 0x1)
    /tmp/src/bigseqkit-cli/helper.go:193 +0x105
main.getFileListFromArgsAndFile(0xc000722390?, {0xc000344720, 0x3, 0x3}, 0x0?, {0x12b5059?, 0xd7e260?}, 0x40?)
    /tmp/src/bigseqkit-cli/helper.go:231 +0x79
main.readSeqs(0xc0007280e8?, {0xc000344720?, 0x2?, 0x0?}, 0x4?)
    /tmp/src/bigseqkit-cli/helper.go:56 +0x9a
main.ignisDriver(0x108f840?, {0xc000344720, 0x3, 0x3}, 0x1305858)
    /tmp/src/bigseqkit-cli/helper.go:104 +0x315
main.init.17.func1.1(0xc000348000?, {0xc000344720?, 0x3?, 0x3?})
    /tmp/src/bigseqkit-cli/seq.go:48 +0x25
github.com/spf13/cobra.(*Command).execute(0xc000348000, {0xc0003446f0, 0x3, 0x3})
    /root/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:860 +0x663
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000d4c80)
    /root/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:974 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
    /root/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:902
main.main()
    /tmp/src/bigseqkit-cli/main.go:4 +0x1e
Mar 12, 2024 02:14:36 <WARN> org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:159): Error closing output stream.
java.net.SocketException: Socket closed
    at sun.nio.ch.NioSocketImpl.ensureOpenAndConnected(Unknown Source) ~[?:?]
    at sun.nio.ch.NioSocketImpl.beginWrite(Unknown Source) ~[?:?]
    at sun.nio.ch.NioSocketImpl.implWrite(Unknown Source) ~[?:?]
    at sun.nio.ch.NioSocketImpl.write(Unknown Source) ~[?:?]
    at sun.nio.ch.NioSocketImpl$2.write(Unknown Source) ~[?:?]
    at java.net.Socket$SocketOutputStream.write(Unknown Source) ~[?:?]
    at java.io.BufferedOutputStream.flushBuffer(Unknown Source) ~[?:?]
    at java.io.BufferedOutputStream.flush(Unknown Source) ~[?:?]
    at java.io.FilterOutputStream.close(Unknown Source) ~[?:?]
    at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:157) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.transport.TSocket.close(TSocket.java:252) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.transport.TZlibTransport.close(TZlibTransport.java:83) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:267) [libthrift-0.15.0.jar:0.15.0]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
    at java.lang.Thread.run(Unknown Source) [?:?]
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.services.IBackendServiceImpl.stopAll(IBackendServiceImpl.java:86): Stopping Backend server
Mar 12, 2024 02:14:36 <WARN> org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:159): Error closing output stream.
java.io.IOException: org.apache.thrift.transport.TTransportException: Cannot write to null outputStream
    at org.apache.thrift.transport.TTransportOutputStream.write(TZlibTransport.java:139) ~[libthrift-0.15.0.jar:0.15.0]
    at java.util.zip.DeflaterOutputStream.deflate(Unknown Source) ~[?:?]
    at java.util.zip.DeflaterOutputStream.finish(Unknown Source) ~[?:?]
    at java.util.zip.DeflaterOutputStream.close(Unknown Source) ~[?:?]
    at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:157) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.transport.TZlibTransport.close(TZlibTransport.java:81) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:270) [libthrift-0.15.0.jar:0.15.0]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
    at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: org.apache.thrift.transport.TTransportException: Cannot write to null outputStream
    at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:194) ~[libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.transport.TTransportOutputStream.write(TZlibTransport.java:137) ~[libthrift-0.15.0.jar:0.15.0]
    ... 9 more
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.services.IBackendServiceImpl.stopHealthServer(IBackendServiceImpl.java:127): Backend health server stopped
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.services.IBackendServiceImpl.start(IBackendServiceImpl.java:71): Backend server stopped
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.cluster.IClusterDestroyHelper.destroy(IClusterDestroyHelper.java:41): Cluster(0): Preparing cluster to destroy
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.worker.IWorkerDestroyHelper.destroy(IWorkerDestroyHelper.java:38): Cluster(0), Worker(0): Preparing worker to destroy
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.cluster.IClusterDestroyHelper.lambda$destroy$0(IClusterDestroyHelper.java:57): Cluster(0): Destroying cluster with 1 containers
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.cluster.IClusterDestroyHelper.lambda$destroy$0(IClusterDestroyHelper.java:59): Cluster(0): Cluster destroyed
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.Main.main(Main.java:176): Backend stopped

[cesarpomar]

Using Docker containers, only the IGNIS_DFS_ID is mounted and set as the working directory. Therefore, you can reference files within that directory using relative paths.

You can use IgnisHPC properties to manually mount other directories using -p just after the script.

...ignis-submit -p ignis.driver.bind.{containerpath}={hostpath} -p ignis.executor.bind.{containerpath}={hostpath}...

We're working on a simpler implementation, but it's not ready yet.