Singularity bind issue

[hkaspersen]

Not sure if it is appropriate to ask this here, as it might be a singularity thing, but I am having some issues with paths and singularity. In my pipeline, I use singularity images, and I tried to make the soft links created by nextflow available to it by adding the singularity.runOptions = "--bind /cluster:/cluster" command in the nextflow.config, as /cluster is the main path on our HPC cluster. However, it does not seem to be able to access the paths or the soft links regardless of this (the program that was run gives the error "file empty"). Am I missing something in the syntax here?
I am using singularity version 3.7.2-1.el7, and nextflow version 21.10.6.
I appreciate any help with this!

[hkaspersen]

Just to add to this: Running singularity run --bind /cluster:/cluster <img> works outside of Nextflow, and the path is available from inside the container. It seems this may be a Nextflow issue perhaps?

[pditommaso]

Symlink can only work if also the target path is mounted in the container

[hkaspersen]

Yes this is true, but why does it work with just /cluster outside the nextflow framework? Do I then need to add the full path to the work directory location?

[pditommaso]

No idea. Inspect the singularity command generated by NF to troubleshoot the cause

[hkaspersen]

This is how it looks like from .command.run:

set +u; env - PATH="$PATH" ${TMP:+SINGULARITYENV_TMP="$TMP"} ${TMPDIR:+SINGULARITYENV_TMPDIR="$TMPDIR"} singularity exec --bind /cluster/work/users/hkaspersen /cluster/work/users/hkaspersen/singularity/biocontainers-parsnp-v1.2dfsg-5-deb_cv1.img /bin/bash -c "cd $PWD; eval $(nxf_container_env); /bin/bash -ue /cluster/work/users/hkaspersen/alppaca_core_genome/0a/53d930bf73ce27c0d7489a31e867f0/.command.sh"

[hkaspersen]

I may have found a solution to this issue: It seems that by downloading the images and using absolute path to them with file:// makes it possible to use --bind. However, when i specify an URL from where nextflow can pull the image, the bind option does not seem to work properly, and the image cannot get access to the file path.