Import control groups file for Singularity to limit container memory consumption

[matt-sd-watson]

Hello,

We have configured a Nextflow pipeline to run through a Singularity container.
Is there an explicit way to be able to use a control group file: https://sylabs.io/guides/3.0/user-guide/cgroups.html within Nextflow to limit Singularity memory consumption? We would like to apply Singularity memory limits on a local run of Nextflow.

I am not able to find any specific configuration scope parameters that allow for use of this file at runtime using Nextflow run:
https://www.nextflow.io/docs/latest/config.html#config-singularity

Any assistance would be appreciated, thank you!

[pditommaso]

This would be desirable, however, Nextflow has not direct support for cpus and memory confinement when using singularity due to the lack of specific CLI options.

However you should still be able to control the use of cgroup file using the containerOptions and specifying the singularity options you are suggesting.

[matt-sd-watson]

Hello Paolo,

Thank you for your response! Since opening the discussion I have seen the option to use the CLI --apply-cgroups in the runOptions as well as containerOptions scopes.
However, applying c groups requires root privileges. Is there a way to specify through Nextflow that you would like the Singularity commands run with sudo privileges? Attempting sudo nextflow run does not seem to work.
Thank you!

[pditommaso]

No, there isn't. All the point of using singularity is to avoid the need for sudo, or it was in the early days. You should consider using a different container runtime in your environment such as Podman.