CI Environments (container or conda env)

[htwangtw]

It will be useful to have a container with FSL for CI. For example, the test for Complex interface is failing, but it is running fine locally for me and Dorta. This should be a good issue to start with for more consistency.

[oesteban]

May these help?
https://github.com/nipreps/fsl
https://github.com/nipreps/afni
https://github.com/nipreps/freesurfer

[htwangtw]

This is great thank you!
cc @djarecka as she is on the case but I will have a look too,

[satra]

@oesteban any reason you are not using neurodocker for it? We are trying to consolidate building various versions of those tools into one place. Those seem handcrafted for a specific use case. Also eventually these pydra repos will use testkraken to test against different versions. So the containers would have to be created for several versions.

[oesteban]

We're entertaining the idea of multi stage docker builds, that's the only reason for it. It's true these hardly allow versioning at the moment

[effigies]

I would suggest we switch to installing FSL with conda, e.g.,

conda install -c anaconda -c https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/public/ -c leej3 -c conda-forge fsl-fnirt

[satra]

is john still maintaining -c leej3 ? or is that the official recommendation from fsl folks? also perhaps use mamba to speed things up.

[effigies]

No. John's AFNI channel is stuck in 2020. I still use it (hence why it slipped into that command I meant just to demo FSL...), but I wouldn't consider it an official channel.

The FSL installer now creates a conda environment, which is how I found this. So I'd consider using it as a conda channel stable but possibly not recommended to general users (as opposed to going through their installer). @pauldmccarthy or @hanayik could probably clarify, though.

[ghisvail]

No. John's AFNI channel is stuck in 2020. I still use it (hence why it slipped into that command I meant just to demo FSL...), but I wouldn't consider it an official channel.

The FSL installer now creates a conda environment, which is how I found this. So I'd consider using it as a conda channel stable but possibly not recommended to general users (as opposed to going through their installer). @pauldmccarthy or @hanayik could probably clarify, though.

I noticed it too during deployment this week.

I reckoned there was a talk at OHBM BrainHack this year about Conda's FSL packages coming soon. I wonder how far away we are from conda installing individual FSL binaries. That would be such an improvement.

[effigies]

I reckoned there was a talk at OHBM BrainHack this year about Conda's FSL packages coming soon.

Yes, that was @hanayik. It was extremely pleasing to hear.

I wonder how far away we are from conda installing individual FSL binaries.

I'm doing it to run tests. I plan to use it in nipreps testing environments, now that neurodebian has stopped packaging FSL. Assuming I'm not outvoted, I'm excited to see how much it slims down fMRIPrep Docker images in 23.0.

[pauldmccarthy]

Hi @effigies, @satra, @ghisvail, etc; yes - we have migrated FSL to being fully conda-based, so advanced users (i.e. you) should be able to use conda/mamba directly to install specific FSL components if you wish. I've been compiling some documentation describing the new system which you might find handy.

We are sourcing packages from our public channel, and from conda-forge (I was not familiar with leej3). We are not sourcing from the anaconda/defaults channel, to avoid any potential issues w.r.t. commercial licensing of FSL (although this is probably not an issue for a small-scale project like FSL, as noted in this reddit comment).

One thing to note is that with a command like:

conda install -c https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/public/ -c conda-forge fsl-fnirt

you will be getting the latest available version of the fsl-fnirt package, which may or may not be the latest version that has been released as part of FSL. There is a chance that we will publish a broken fsl-fnirt package which we only catch after publishing (the bulk of our tests are run by hand off-line). So you may wish to pin to the latest available released version, which will have been through our full test suite. FSL releases are defined at our manifest repository, so to get the fnirt that was released with 6.0.6.1:

conda install -c https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/public/ -c conda-forge fsl-fnirt=2203.0

Another potential issue which you may encounter while we are transitioning to this new system - while the dependencies for all C/C++ projects should be accurate (as they wouldn't be buildable otherwise), there may be some under-/over-specifications in the dependencies for bash/tcl/python scripts. We're planning to fix these on a case-by-case basis, so please report any issues you encounter!

[effigies]

Are the base dependencies pinned so that installing the 6.0.6.1 version of fugue will install the 6.0.6.1 version of all its dependencies, or would we need to pin all of them?

[pauldmccarthy]

@effigies yes, this should be the case, at least for the binary projects (we've been a bit more lenient w.r.t. pinning for the other projects).