scipy from conda, for correct BLAS/LAPACK operations in arm64

[mikibonacci]

This is a problem related to the QE app in arm64, in particular with the aiidalab-qe-vibroscopy plugin. The problem is in the calculation of inelastic neutron scattering spectra, which are computed in euphonic using scipy modules, which are using BLAS/LAPACK. In arm64 there is some broken libraries/linking if scipy is installed from pip, which are fixed if the package is installed from conda (conda-forge).
Otherwise, when INS data are processed, the app gets stuck and cannot compute spectra.

so, we should add conda install scipy==1.13.1 --y somewhere (or the mamba analogous).

I already added this in the aiidalab/qe image, but we should do it here as the user can install the app also just using the aiidalab/full-stack image.

As final comment, to be safe and cover also corner cases in non arm64 machines, I suggest we always install scipy via conda.

[danielhollas]

I agree that it would be better to have scipy pre-installed via conda.
I think this will be covered once we include aiidalab-widgets-base in the base image since it depends on scipy. That might still take some time though. In the meantime, maybe we could install scipy via installing atomic_tools extras from aiida-core package.

Otherwise, when INS data are processed, the app gets stuck and cannot compute spectra.

Can you share more information about the error / what exactly happens?

[mikibonacci]

Not quite sure about the error, what I see on the log (print) is:

 ** On entry to ZHEEVD parameter number 10 had an illegal value
INFO: Zheevd diagonalisation failed with info -10 at q-point 0.000000 0.000000 0.000000

then this blocks the code (euphonic) during the calculation of phonons.

[danielhollas]

And just to be clear, the error goes away when you reinstall scipy with conda, for the exact same inputs?

It would be nice to have the exact steps how to reproduce this.

[mikibonacci]

Hi @danielhollas , exactly, just installing scipy via conda solves the issue, with the same inputs.

I am not sure if we should investigate more on this, or just be happy with this solution (I am). Anyway, here below the instructions to reproduce the problem:

Basically, we need to open the qe app to compute and visualize neutron scattering data, using scipy installed via pip in arm64.
However, is not that simple, as in the code I have some redirect of the standard out of the code (in this way, built-in print statements in the euphonic functions are not shown in the app). So you will just see the app hanging forever[1]

If you have a full stack image, you can:

• install the qe app (and its aiidalab-qe-vibroscopy plugin)
• then download one of the phonons+INS examples (in the download examples page) - like the Silicon phonons+INS.
• then you open the job, go to the vibrational result panel, go to neutron scattering tab and click on "initialise INS data".

The initialisation will just hang forever, and the app then will be stuck in this phase.

[1]To have a print of the error, you need to check in the aiidalab-qe-vibroscopy plugin, I put some enablePrint and blockPrint functions, very dummy functions with obvious purpose.