Define apptainer.cacheDir to be in workDir

[Poshi]

Bug report

Expected behavior and actual behavior

When trying to define the cacheDir in the apptainer scope to be unser the working folder, the workDir does not exist, so it cannot be set up there.
By default, Nextflow puts the cache in $workDir/singularity, but it also issues a warning about that:

WARN: Apptainer cache directory has not been defined -- Remote image will be stored in the path: /home/user/pipeline/work/singularity -- Use the environment variable NXF_APPTAINER_CACHEDIR to specify a different location

I'm trying to get rid of the warning while keeping the folder under the working dir by making that location explicit.

Steps to reproduce the problem

Use a configuration that includes the following snippet:

apptainer {
    enabled = true
    cacheDir = "${workDir}/apptainer"
}

Program output

ERROR ~ Unknown config attribute `apptainer.workDir` -- check config file: /home/user/pipeline/nextflow.config

Environment

• Nextflow version: 23.04.3
• Java version: java 11.0.15 2022-04-19 LTS
• Operating system: CentOS Linux 7 (Core)/3.10.0-1160.88.1.el7.x86_64
• Bash version: GNU bash, version 4.2.46(2)-release

[ewels]

Note that the warning message about the Apptainer cache directory is not dependent on setting cacheDir. It would be great if we could update Nextflow to run without this warning coming up, I suspect it's a Singularity / Apptainer thing?

@Poshi - what version of Apptainer are you using?

[Poshi]

My apptainer version is:
apptainer version 1.1.6-1.el7

I don't think it is an apptainer message, and I don't agree that the setting and the warning are not related:

1. If it were an apptainer thing, the warning would not tell me to set the NXF_APPTAINER_CACHEDIR environment variable. This is clearly a Nextflow message.
2. If the message ("Apptainer cache directory has not been defined") does not refer to this variable, then which is the variable I should use to define the Apptainer cache directory? Sincerely, apptainer.cacheDir looks like the proper answer. And me, as a user, would expect not to see that warning if I set up that variable.

In any case, you are the experts on Nextflow, these are just my thoughts :-) Thanks for looking into this!

[bentsherman]

The warning comes from Nextflow:

nextflow/modules/nextflow/src/main/groovy/nextflow/container/SingularityCache.groovy

Lines 262 to 263 in fe5bea9

	if( missingCacheDir )
	log.warn1 "${appName} cache directory has not been defined -- Remote image will be stored in the path: $targetPath.parent -- Use the environment variable NXF_${envPrefix}_CACHEDIR to specify a different location"

The best practice here is to set NXF_APPTAINER_CACHEDIR or apptainer.cacheDir to a more permanent location than the work directory. I usually set it to something like $HOME/apptainer-cache so that all of my Nextflow runs can share the same cache.

As for your original issue, I guess the workDir variable is exposed in the pipeline script but not the config file.

[ewels]

Ahh sorry then I misread / misunderstood - so actually the warning is deliberate to try to prevent users from repeatedly downloading the same images again and again? Makes sense 👍🏻

[Poshi]

Well, on one side, I'd like to be able to mimmick the nextflow default while avoiding the warning. On another side, I was thinking on setting it to $workDir/../apptainer. That way I will be free to clean up the work folder from time to time without killing the apptainer cache, but the cache will be a companion of the work folder. Using relative paths makes it easy to reuse the configuration as-is without having to personalize it for every location where it runs. And to me, the source of truth for the work folder is the one that comes from Nextflow itself. Setting an environment variable means that I have to keep the work dir configuration in more than one place, and that I should switch it for every workflow if I don't want an unified cache (why I could not want an unified cache is another question).

[bentsherman]

Then I would just set apptainer.cacheDir = 'apptainer', assuming that you are using the default work directory, that will put the cache directory alongside the work directory.

[Poshi]

:-O
So easy. I didn't knew that the work dir was the current dir. It could have been the project dir, or the launch dir; that's why I tried to set it to $workDir, because it was the only way I knew to se it to this folder.
It is documented somewhere which is the folder where the code runs? I mean: for each parameter that needs a folder as input, there's some documentation about the meaning of .?

[bentsherman]

Relative paths are resolved against the directory where you launched nextflow. Or you can set NXF_FILE_ROOT to change the base paths for relative paths.

[Poshi]

That sounds like a workaround that won't work in all the cases.
Anything that relies on environment variables implies having the configuration split in different locations (file and environment) and potentially, having to duplicate configuration values (or prefixes, like when one variable depends on another one). That's something I'm trying to avoid.
With your workaround, it will work as long as the work dir is a subfolder of the launch dir, but in the case of setting the workdir to another location (either by environment or thru config file), the cache would still be in the launch dir.

[Poshi]

After all, I assume there's no way to avoid the warning message while keeping the same behavior, right? And there's no intention of solving it as this is no longer an issue, but a discussion, right?