CLI to get modules installation path

[marcodelapierre]

Hi @vsoch,

yesterday I was starting to think how the container story for the talk in a month might look like; in my mind the central topic is user experience.
So, provided the dependencies are available (container runtime, module system, Python), I started to picture myself showing how to get up and running with container modules in the simplest possible way, and ...

pip install singularity-hpc
shpc install vanessa/salad
module use $( shpc QUOKKA )
module load vanessa/salad
salad

Hold on a minute....have I just said QUOKKA ?!

Yes, I have .. because I have realised that QUOKKA would be a nice addition to the SHPC CLI functionalities.
As you see, QUOKKA is a way for SHPC to provide the shell with the path where the modulefiles are installed.

We have this for views:

shpc view get <viewname>

So that one can do

module use $( shpc view get <viewname> )

But we don't have this for non view configurations.

My temptation would be to be consistent with views and just introduce shpc get, however this is currently being used to provide the path to the SIF image for a given registry. How should we proceed?

Note that right now one can get the path to a SIF image by means of:

module load <registry>
<registry>-container

So I am tempted to say we could deprecate this old command meaning, and update to the suggestion above.
Or we could update the current get CLI to become e.g. get_sif or similar.

Last point: right now the full CLI looks like:

module use $( shpc view get <viewname>)
module use $( shpc QUOKKA )

Should we update it so that the shpc CLI embeds the module use bit? (is it even doable from Python, as module is a shell function?) We can leave as is, just a thought.

What are your thoughts? Thank you!

[vsoch]

Wait I can't tell if QUOKKA is a joke or an actual command? You are saying you want an equivalent of shpc get <modulename> but not for the container? What's a QUOKKA?

[marcodelapierre]

I am glad you asked!!

https://bestlifeonline.com/quokka/

[marcodelapierre]

In this context, it is a command that would basically print the SHPC internal $module_base out to the shell

(fueled by Quokka energy, I forgot to state this in the above description!)

[marcodelapierre]

please refresh your page, I have done some in-comment bug fixes!

[marcodelapierre]

So I am asking for an equivalent of shpc view get <viewname, but not for the views, but rather for the generic $module_base.

[marcodelapierre]

That way it is possible to get shpc set up in a very concise and elegant way, as per my very first code snippet :-)

[marcodelapierre]

(I know, by now you will be lost in quokka cuteness!!!)

[vsoch]

(I know, by now you will be lost in quokka cuteness!!!)

omg, indeed they are adorable!

If this is for a specific module, what about shpc module get <name> ?

[marcodelapierre]

It is for the root dir of the module tree:

$ find $(pwd)
/home/ubuntu/singularity-hpc/modules
/home/ubuntu/singularity-hpc/modules/vanessa
/home/ubuntu/singularity-hpc/modules/vanessa/salad
/home/ubuntu/singularity-hpc/modules/vanessa/salad/.version
/home/ubuntu/singularity-hpc/modules/vanessa/salad/latest
/home/ubuntu/singularity-hpc/modules/vanessa/salad/latest/module.lua
/home/ubuntu/singularity-hpc/modules/python
/home/ubuntu/singularity-hpc/modules/python/3.9.2-slim
/home/ubuntu/singularity-hpc/modules/python/3.9.2-slim/module.lua
/home/ubuntu/singularity-hpc/modules/python/.version

[marcodelapierre]

In this example, we need /home/ubuntu/singularity-hpc/modules, so as to run (in shell or better in .bashrc):

$ module use /home/ubuntu/singularity-hpc/modules

To enable the module tree within the module system.

[marcodelapierre]

The end result being:

$ module avail

------------------------------------ /home/ubuntu/singularity-hpc/modules ------------------------------------
   python/3.9.2-slim/module    vanessa/salad/latest/module

[vsoch]

So you want shpc to do something the module software already does?

[marcodelapierre]

Nope, look at this:

module use $( shpc view get <viewname>)
module use $( shpc QUOKKA )

I want SHPC to be able to spit out the path that module use needs as input.

[vsoch]

We already have:

$ shpc config get module_base
module_base                    /home/vanessa/Desktop/Code/shpc/modules

So you could probably just add a flag that says "only give me the actual value" fairly easily.

[vsoch]

We could even make the single value the default, and then have the extra identifier provided with a flag (that probably is more expected).