Singularity container creation fails with process.scratch=true

[rfenouil]

Bug report

When trying to run the nfcore/rnaseq pipeline with following command:
nextflow run nf-core/rnaseq -profile test,singularity -c config_process

with config_process containing:

singularity.engineOptions = "--debug"
process.scratch = true

The execution fails with the following error message:

  DEBUG   [U=1312005958,P=3696791]mountGeneric()                Mounting /tmp/nxf.d50TB8mONY to /usr/local/var/singularity/mnt/session/final/tmp/nxf.d50TB8mONY
  FATAL   [U=1312005958,P=3696791]Master()                      container creation failed: mount /tmp/nxf.d50TB8mONY->/tmp/nxf.d50TB8mONY error: while mounting /tmp/nxf.d50TB8mONY: destination /tmp/nxf.d50TB8mONY doesn't exist in container

Expected behavior and actual behavior

I believe my network shared filesystem has some limitations that prevent the proper execution of salmon_quant (crashes, not sure why actually).
When using the option -w ~/tempFolder execution succeeds, so I figured I could set process.scratch = true in a config file to avoid creating the temp/work folder manually every time.

Environment

• Nextflow version: 21.10.6 build 5660
• Java version: openjdk version "17.0.1" 2021-10-19
• Operating system: Ubuntu 20.04.2 LTS
• Bash version: GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)

[dodslaser]

Is there a solution to this problem? I am also running into this when using process.scratch = true with singularity/apptainer.

I found this note in the apptainer docs that could be relevant:
https://docs.sylabs.io/guides/3.5/user-guide/bind_paths_and_mounts.html#a-note-on-using-bind-with-the-writable-flag