Add jstack command

[joshmoore]

Initial work on a jstack command which allows omero admin jstack to determine the pid of
the given process ("Blitz-0" by default) and pass that to the local jstack tool.

Example

omero admin jstack
Loading jstack for Blitz-0 (1160)
stdout:
2024-01-22 14:58:04
Full thread dump OpenJDK 64-Bit Server VM (11.0.21+9-LTS mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007fffac000c30, length=48, elements={
0x00007ffff8027800, 0x00007ffff840e000, 0x00007ffff8410800, 0x00007ffff841a000,
0x00007ffff841c000, 0x00007ffff841e000, 0x00007ffff8420000, 0x00007ffff8422000,
0x00007ffff846f000, 0x00007ffff8d4c800, 0x00007ffff8e99800, 0x00007ffff927c800,
0x00007ffff9285800, 0x00007ffff946d800, 0x00007ffff94ff800, 0x00007ffff959f800,
0x00007ffff9dc4000, 0x00007ffff9dc7000, 0x00007fff34007000, 0x00007fff34003800,
0x00007fff34005000, 0x00007fff3400b800, 0x00007ffff9390800, 0x00007ffff988b000,
0x00007ffff989d000, 0x00007fff3400d000, 0x00007ffffa205000, 0x00007ffed4001800,
0x00007ffef0059800, 0x00007fff8c002800, 0x00007ffed4002800, 0x00007fff8c003800,
0x00007fff50dd9800, 0x00007fff88a7b800, 0x00007ffed4004800, 0x00007ffec0001000,
0x00007fff50dda800, 0x00007ffec8001800, 0x00007ffed4005800, 0x00007ffefc01d000,
0x00007ffec0002800, 0x00007ffee40cb800, 0x00007fff50ddb000, 0x00007fff00007000,
0x00007ffec8002800, 0x00007ffef8029800, 0x00007fffac001000, 0x00007ffebc001000
}

"main" #1 prio=5 os_prio=0 cpu=11314.58ms elapsed=10209.15s tid=0x00007ffff8027800 nid=0x4a0 in Object.wait()  [0x00007ffffe961000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.21/Native Method)
	- waiting on <0x000000066437b428> (a IceInternal.ObjectAdapterFactory)
	at java.lang.Object.wait(java.base@11.0.21/Object.java:328)
	at IceInternal.ObjectAdapterFactory.waitForShutdown(ObjectAdapterFactory.java:63)
	- waiting to re-lock in wait() <0x000000066437b428> (a IceInternal.ObjectAdapterFactory)
	at Ice.CommunicatorI.waitForShutdown(CommunicatorI.java:32)
	at ome.services.blitz.Entry.start(Entry.java:202)
	at ome.services.blitz.Entry.main(Entry.java:146)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=6.46ms elapsed=10209.04s tid=0x00007ffff840e000 nid=0x4be waiting on condition  [0x00007fffe024d000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@11.0.21/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@11.0.21/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@11.0.21/Reference.java:213)

Possible features:

• better handling of missing jstack command.
• write to a file
• use a patterned file name
• periodically repeat

cc: @pwalczysko

[sbesson]

@pwalczysko is this something you'd like to evaluate functionally for possible inclusion in the next OMERO.py release?

[will-moore]

Tried testing on merge-ci with the following commands...

[wmoore@idr1-slot2 ~]$ sudo su
[root@idr1-slot2 wmoore]# docker exec -it merge-ci-omero-1 bash
bash-5.1$ source /home/omero/workspace/OMERO-server/.venv3/bin/activate
(.venv3) bash-5.1$ which omero
/home/omero/workspace/OMERO-server/.venv3/bin/omero
(.venv3) bash-5.1$ omero admin jstack
FATAL: OMERO directory does not exist: /home/omero/workspace/OMERO-server/.venv3/lib64/etc/grid

[joshmoore]

That final error is the standard "Thou shalt have OMERODIR configured" message, i.e. unrelated to jstack.

[will-moore]

So I need to know what value to set OMERODIR to on merge-ci. Any way to find this out?

[joshmoore]

Since you are in the docker container, it'll be something (close to)

/home/omero/workspace/OMERO-server/OMERO.server

[will-moore]

Trying with OMERODIR...

[wmoore@idr1-slot2 ~]$ sudo su
[root@idr1-slot2 wmoore]# docker ps
[root@idr1-slot2 wmoore]# docker exec -it devspace-omero-1 bash
bash-5.1$ source /home/omero/workspace/OMERO-server/.venv3/bin/activate
(.venv3) bash-5.1$ export OMERODIR=/home/omero/workspace/OMERO-server/OMERO.server
(.venv3) bash-5.1$ omero admin jstack
Loading jstack for Blitz-0 (17751)

After about 10 minutes it timed out (connection lost).