Bug report related to e4s-cl init

[thomas-robinson]

I am running on a cloud cluster set up on a parallel works platform. I was trying to run e4s-cl init, but I ran into an error, and after looking at documentation, I set up a profile

e4s-cl profile detect -p pw srun --mpi=pmi2 -N 3 -n 9  ./c_hello.x 

Then I tried to run e4s-cl with my profile and got an error saying I should report a bug here

$ e4s-cl init --launcher srun --backend singularity --profile pw
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
Simulating MPI execution using:
Compiler: /apps/oneapi/mpi/2021.5.1/bin/mpicc
Launcher /usr/bin/srun
The target launcher /usr/bin/srun uses a single host by default, which may tamper with the library discovery. Consider running `e4s-cl profile detect` using mpirun specifying multiple hosts.
Failed to list opened files of /tmp/tmpgwhuuqtz: [Errno 2] No such file or directory
srun: error: thomasrobinson-fv3-00009-1-0007: task 0: Exited with exit code 156
Failed to determine necessary libraries: program exited with code 156
MPI execution failed.
An unexpected KeyError exception was raised:

2

Traceback (most recent call last):
  File "/contrib/bin/e4s-cl", line 12, in <module>
    sys.exit(cli_main_cmd.main(sys.argv[1:]))
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/commands/__main__.py", line 200, in main
    return cli.execute_command([cmd], cmd_args)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/__init__.py", line 314, in execute_command
    return find_command(cmd).main(cmd_args)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/commands/init.py", line 436, in main
    controller.update({'name': requested_name}, profile.eid)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/model/profile.py", line 102, in update
    super().update(data, keys)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/mvc/controller.py", line 236, in update
    database.update(data, keys, table_name=self.model.name)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cf/storage/local_file.py", line 459, in update
    table.update(fields, doc_ids=[keys])
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 340, in update
    self._update_table(updater)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 709, in _update_table
    updater(table)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 337, in updater
    perform_update(table, doc_id)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 326, in perform_update
    table[doc_id].update(fields)
KeyError: 2

This is a bug in E4S Container Launcher.
Please raise an issue on Github with the contents of '/home/Thomas.Robinson/.local/e4s_cl/logs/debug_log'.

Here's the contents of the file

$ cat /home/Thomas.Robinson/.local/e4s_cl/logs/debug_log
[Debug root:491] Unlink /home/Thomas.Robinson/.local/e4s_cl/logs/latest failed: [Errno 2] No such file or directory: '/home/Thomas.Robinson/.local/e4s_cl/logs/latest'
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 12:21:03.378903
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib
Terminal Size     : 112x26
Frozen            : False
Log ID            : fd02aa3ff3e94da26c65add3747862b90677a8053e9498c33c7214f7cf2e804b
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cf.storage.local_file:146] Initialized system filesystem prefix '/contrib/system'
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cf.storage.local_file:50] '/contrib/system/system.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized system database '/contrib/system/system.json'
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
[Debug e4s_cl.cli.commands.init:77] e4s-cl init args: Namespace(launcher='/usr/bin/srun', backend='singularity')
[Debug e4s_cl.cli.commands.init:185] MPI vendor not supported by precompiled binary initialisation
Proceeding with legacy initialisation
[Debug e4s_cl.cli.commands.init:121] Compiling with: '/apps/oneapi/mpi/2021.5.1/bin/mpicc -o /tmp/tmpkv6f6q4s -lm /tmp/tmpwaazorcm.c'
Simulating MPI execution using:
Compiler: /apps/oneapi/mpi/2021.5.1/bin/mpicc
Launcher /usr/bin/srun
The target launcher /usr/bin/srun uses a single host by default, which may tamper with the library discovery. Consider
running `e4s-cl profile detect` using mpirun specifying multiple hosts.
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name=None, cmd=['/usr/bin/srun', '/tmp/tmpkv6f6q4s'])
Failed to determine necessary libraries: program exited with code 156
MPI execution failed.
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 12:27:00.260572
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib
Terminal Size     : 112x26
Frozen            : False
Log ID            : 2ed65e431bfbcab3168ae4f4590fc2a6ccc1448853abf8174ff0fced595e54c8
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cf.storage.local_file:50] '/contrib/system/system.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized system database '/contrib/system/system.json'
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
[Debug e4s_cl.cli.commands.init:77] e4s-cl init args: Namespace(launcher='/usr/bin/srun', backend='singularity')
[Debug e4s_cl.cli.commands.init:185] MPI vendor not supported by precompiled binary initialisation
Proceeding with legacy initialisation
[Debug e4s_cl.cli.commands.init:121] Compiling with: '/apps/oneapi/mpi/2021.5.1/bin/mpicc -o /tmp/tmp50exhn5p -lm /tmp/tmprztzimz9.c'
Simulating MPI execution using:
Compiler: /apps/oneapi/mpi/2021.5.1/bin/mpicc
Launcher /usr/bin/srun
The target launcher /usr/bin/srun uses a single host by default, which may tamper with the library discovery. Consider
running `e4s-cl profile detect` using mpirun specifying multiple hosts.
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name=None, cmd=['/usr/bin/srun', '/tmp/tmp50exhn5p'])
Failed to determine necessary libraries: program exited with code 156
MPI execution failed.
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 12:43:09.562829
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib
Terminal Size     : 112x26
Frozen            : False
Log ID            : 4000e36d81b7ad1886b0430b9bccdc4f973f7d488b98f11a1528e1d2f8009167
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cf.storage.local_file:50] '/contrib/system/system.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized system database '/contrib/system/system.json'
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
[Debug e4s_cl.cli.commands.init:77] e4s-cl init args: Namespace(launcher='/usr/bin/srun', backend='singularity')
[Debug e4s_cl.cli.commands.init:185] MPI vendor not supported by precompiled binary initialisation
Proceeding with legacy initialisation
[Debug e4s_cl.cli.commands.init:121] Compiling with: '/apps/oneapi/mpi/2021.5.1/bin/mpicc -o /tmp/tmp3k608a5f -lm /tmp/tmp7wm2qne0.c'
Simulating MPI execution using:
Compiler: /apps/oneapi/mpi/2021.5.1/bin/mpicc
Launcher /usr/bin/srun
The target launcher /usr/bin/srun uses a single host by default, which may tamper with the library discovery. Consider
running `e4s-cl profile detect` using mpirun specifying multiple hosts.
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name=None, cmd=['/usr/bin/srun', '/tmp/tmp3k608a5f'])
Failed to determine necessary libraries: program exited with code 156
MPI execution failed.
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 12:48:33.328930
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib
Terminal Size     : 112x26
Frozen            : False
Log ID            : 80bee8cfe4305db665a79590bafc927ad89865710fad0e93acbc59a26de0f525
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=[])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name=None, cmd=[])
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 12:49:27.375315
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib
Terminal Size     : 112x26
Frozen            : False
Log ID            : 1370b8b30920376a8de2c875365a7c79ec2168c1e968882b4dbe18d8f5fa6064
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cf.storage.local_file:50] '/contrib/system/system.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized system database '/contrib/system/system.json'
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
[Debug e4s_cl.cli.commands.init:77] e4s-cl init args: Namespace(launcher='/usr/bin/srun', backend='singularity')
[Debug e4s_cl.cli.commands.init:185] MPI vendor not supported by precompiled binary initialisation
Proceeding with legacy initialisation
[Debug e4s_cl.cli.commands.init:121] Compiling with: '/apps/oneapi/mpi/2021.5.1/bin/mpicc -o /tmp/tmp5tum_jh1 -lm /tmp/tmpxlk4p3nz.c'
Simulating MPI execution using:
Compiler: /apps/oneapi/mpi/2021.5.1/bin/mpicc
Launcher /usr/bin/srun
The target launcher /usr/bin/srun uses a single host by default, which may tamper with the library discovery. Consider
running `e4s-cl profile detect` using mpirun specifying multiple hosts.
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name=None, cmd=['/usr/bin/srun', '/tmp/tmp5tum_jh1'])
Failed to determine necessary libraries: program exited with code 156
MPI execution failed.
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 12:56:16.392146
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib
Terminal Size     : 112x26
Frozen            : False
Log ID            : b68d0d3b050bc18c09ed12970cec9dcf9d9b8c3816b2c77b4a66cce73167bbd7
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='init', options=['--launcher', 'srun -N6', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='init', options=['--launcher', 'srun -N6', '--backend', 'singularity'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cf.storage.local_file:50] '/contrib/system/system.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized system database '/contrib/system/system.json'
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
usage: e4s-cl init [-h] [--system machine] [--launcher launcher]
                   [--launcher_args launcher_args] [--mpi /path/to/mpi]
                   [--source script] [--image /path/to/image] [--backend technology]
                   [--profile profile_name] [--wi4mpi path] [--wi4mpi_options opts]
e4s-cl init: error: argument --launcher: Launcher argument 'srun -N6' could not be resolved to a binary
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 12:59:12.937213
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : f8be31d20e3ad8fe2ea5d3b133dbc6f021d2492d714ae70f3b07940e11a1dfd5
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '-N', '3', '-n', '3', './a.out'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '-N', '3', '-n', '3', './a.out'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=['-p', 'pw', 'srun', '-N', '3', '-n', '3', './a.out'])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name='pw', cmd=['srun', '-N', '3', '-n', '3', './a.out'])
Failed to determine necessary libraries: program exited with code 156
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 13:16:57.733090
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : 862ab45a8cf14653ac482ca977a491d90bde64fae9b391f1ad1cb12c9908efb4
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '3', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '3', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=['-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '3', './c_hello.x'])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name='pw', cmd=['srun', '--mpi=pmi2', '-N', '3', '-n', '3', './c_hello.x'])
Received keyboard interrupt. Exiting.
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 13:17:22.943267
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : faa959952f2b5fbb27253e987303baba50cf930b7a424fb5e1bd79974c39def9
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '164', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '164', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=['-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '164', './c_hello.x'])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name='pw', cmd=['srun', '--mpi=pmi2', '-N', '3', '-n', '164', './c_hello.x'])
Failed to determine necessary libraries: program exited with code 1
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 13:17:39.959255
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : d6737d8a717d7421c5ca98fb8ab422465f16c489e0b2e351b881ee905c8383b2
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-n', '164', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-n', '164', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=['-p', 'pw', 'srun', '--mpi=pmi2', '-n', '164', './c_hello.x'])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name='pw', cmd=['srun', '--mpi=pmi2', '-n', '164', './c_hello.x'])
Received keyboard interrupt. Exiting.
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 13:18:26.050153
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : 4e6b9d8703ee647fa200ec47dda7a9dd807aee6690e8e05a5545d8aed37b6413
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '96', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '96', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=['-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '96', './c_hello.x'])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name='pw', cmd=['srun', '--mpi=pmi2', '-N', '3', '-n', '96', './c_hello.x'])
Failed to determine necessary libraries: program exited with code 244
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 13:23:17.684747
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : 063a179af243a7467b11d2e2a95c37ecdd23a59cbb46420f8fd007f33addc0d9
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '96', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '96', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=['-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '96', './c_hello.x'])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name='pw', cmd=['srun', '--mpi=pmi2', '-N', '3', '-n', '96', './c_hello.x'])
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 13:28:35.830914
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : 50f37f205e994b6ca01871c9c7064573903390a37ec908596a7e2c811fffe009
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '9', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '9', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=['-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '9', './c_hello.x'])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name='pw', cmd=['srun', '--mpi=pmi2', '-N', '3', '-n', '9', './c_hello.x'])
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 13:30:16.899563
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : d274bdec7f5d2ed4e9c1150e2ddd1f7930fdcf021231c23c2631ce57a563c353
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '9', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='profile', options=['detect', '-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '9', './c_hello.x'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cli.commands.profile:77] e4s-cl profile args: Namespace(subcommand='detect', options=['-p', 'pw', 'srun', '--mpi=pmi2', '-N', '3', '-n', '9', './c_hello.x'])
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name='pw', cmd=['srun', '--mpi=pmi2', '-N', '3', '-n', '9', './c_hello.x'])
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
[Debug root:503] 
################################################################################################################
E4S CONTAINER LAUNCHER LOGGING INITIALIZED

Timestamp         : 2022-05-27 13:30:52.826689
Hostname          : mgmt-thomasrobinson-fv3-00009
Platform          : Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
Version           : 1.0rc6
Python Version    : 3.9.12
Working Directory : /contrib/test
Terminal Size     : 112x26
Frozen            : False
Log ID            : dae948381ffebe02644147d6fbd13a77c094c577909aa8e5038c6e917eed7ccc
################################################################################################################

[Debug e4s_cl.cf.wi4mpi:26] Getting WI4MPI root failed
[Debug e4s_cl.cli.commands.__main__:77] e4s-cl args: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity', '--profile', 'pw'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:195] Arguments: Namespace(command='init', options=['--launcher', 'srun', '--backend', 'singularity', '--profile', 'pw'], dry_run=None)
[Debug e4s_cl.cli.commands.__main__:196] Verbosity level: INFO
[Debug e4s_cl.cf.storage.local_file:50] '/contrib/system/system.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized system database '/contrib/system/system.json'
[Debug e4s_cl.cf.storage.local_file:50] '/home/Thomas.Robinson/.local/e4s_cl/user.json' opened read-write
[Debug e4s_cl.cf.storage.local_file:170] Initialized user database '/home/Thomas.Robinson/.local/e4s_cl/user.json'
[Debug e4s_cl.cli.commands.init:77] e4s-cl init args: Namespace(launcher='/usr/bin/srun', backend='singularity', profile_name='pw')
[Debug e4s_cl.cli.commands.init:185] MPI vendor not supported by precompiled binary initialisation
Proceeding with legacy initialisation
[Debug e4s_cl.cli.commands.init:121] Compiling with: '/apps/oneapi/mpi/2021.5.1/bin/mpicc -o /tmp/tmpgwhuuqtz -lm /tmp/tmphp09k8es.c'
Simulating MPI execution using:
Compiler: /apps/oneapi/mpi/2021.5.1/bin/mpicc
Launcher /usr/bin/srun
The target launcher /usr/bin/srun uses a single host by default, which may tamper with the library discovery. Consider
running `e4s-cl profile detect` using mpirun specifying multiple hosts.
[Debug e4s_cl.cli.commands.profile.detect:77] e4s-cl profile detect args: Namespace(profile_name=None, cmd=['/usr/bin/srun', '/tmp/tmpgwhuuqtz'])
Failed to determine necessary libraries: program exited with code 156
MPI execution failed.
[Debug e4s_cl.error:168] Traceback (most recent call last):
  File "/contrib/bin/e4s-cl", line 12, in <module>
    sys.exit(cli_main_cmd.main(sys.argv[1:]))
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/commands/__main__.py", line 200, in main
    return cli.execute_command([cmd], cmd_args)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/__init__.py", line 314, in execute_command
    return find_command(cmd).main(cmd_args)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/commands/init.py", line 436, in main
    controller.update({'name': requested_name}, profile.eid)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/model/profile.py", line 102, in update
    super().update(data, keys)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/mvc/controller.py", line 236, in update
    database.update(data, keys, table_name=self.model.name)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cf/storage/local_file.py", line 459, in update
    table.update(fields, doc_ids=[keys])
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 340, in update
    self._update_table(updater)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 709, in _update_table
    updater(table)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 337, in updater
    perform_update(table, doc_id)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 326, in perform_update
    table[doc_id].update(fields)
KeyError: 2

An unexpected KeyError exception was raised:

2

Traceback (most recent call last):
  File "/contrib/bin/e4s-cl", line 12, in <module>
    sys.exit(cli_main_cmd.main(sys.argv[1:]))
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/commands/__main__.py", line 200, in main
    return cli.execute_command([cmd], cmd_args)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/__init__.py", line 314, in execute_command
    return find_command(cmd).main(cmd_args)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cli/commands/init.py", line 436, in main
    controller.update({'name': requested_name}, profile.eid)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/model/profile.py", line 102, in update
    super().update(data, keys)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/mvc/controller.py", line 236, in update
    database.update(data, keys, table_name=self.model.name)
  File "/contrib/conda/lib/python3.9/site-packages/e4s_cl/cf/storage/local_file.py", line 459, in update
    table.update(fields, doc_ids=[keys])
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 340, in update
    self._update_table(updater)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 709, in _update_table
    updater(table)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 337, in updater
    perform_update(table, doc_id)
  File "/contrib/conda/lib/python3.9/site-packages/tinydb/table.py", line 326, in perform_update
    table[doc_id].update(fields)
KeyError: 2

This is a bug in E4S Container Launcher.
Please raise an issue on Github with the contents of '/home/Thomas.Robinson/.local/e4s_cl/logs/debug_log'.

Let me know if there's any other information you need

[spoutn1k]

Hi Thomas !

What the init does is wrap profile detect and makes assumptions about the system that are prone to errors. This KeyError exception should however not happen, and we'll take a look at this.

Did the profile detect command succeed ? What error did you encounter while running the profile ? The error you show is from an init command, that should just create a profile.