Add support for Singularity containers

[adthrasher]

The Singularity container platform is an alternative container engine to Docker. It has the advantage of being supported in most HPC environments, where Docker is not supported due to requiring root-level permissions. The latest documentation for Singularity is here.

[claymcleod]

Am I correct that Singularity doesn't have an API and, instead, is only accessible via that command line? If so, I think relying on the generic backend to interact with Singularity is going to be the best bet.

[adthrasher]

@claymcleod - Can you expound a bit on how that works? Are you suggesting that there would not be Singularity support within Crankshaft?

[claymcleod]

I'm just saying that, if there is no API to interact with, the paradigm of executing a subprocess and parsing the response is generally covered by the generic executor (i.e., you can configure the generic executor to use singularity). Unless there is a really specific reason as to why we'd need like a dedicated Singularity backend, it doesn't make sense to me to duplicate the "call a subprocess and return the result" functionality in a different executor.

[mbhall88]

Note: Singularity has become Apptainer a while ago now

Support for Apptainer (Singularity) would be sensational. Or, if support is not possible, at least a detail example/documentation of how to use it with the generic backend would be most appreciated.

I am interested in migrating from snakemake to sprocket/crankshaft and apptainer/conda environments are a foundation of most of my tasks. So support for both/either of these would definitely make it an easy switch.

[claymcleod]

That's great!

I ended up eating my words a bit here, and we did end up deciding that we're going to have to do bespoke backends for most things (e.g., https://github.com/stjude-rust-labs/wdl/pull/600)—in the beginning, we'll be focused on Apptainer + LSF, but we do plan to make these pieces work with more environments in the future.