`augur` not usable in latest osx-arm64 package on macOS ≥15.4

[victorlin]

Running augur -h with nextstrain-base's osx-arm64 version of 20250311T191147Z shows a cryptic error:

ImportError: Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python interpreter from there.

click for full error

Traceback (most recent call last):
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/__init__.py", line 24, in <module>
    from . import multiarray
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/multiarray.py", line 10, in <module>
    from . import overrides
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/overrides.py", line 8, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/_multiarray_umath.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libgfortran.5.dylib
  Referenced from: <CAA510D4-816A-34E5-9021-485EF5D56159> /Users/victor/.micromamba/envs/nextstrain-ambient/lib/libopenblas.0.dylib
  Reason: tried: '/Users/victor/.micromamba/envs/nextstrain-ambient/lib/libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/lib/libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/../../../../libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/../../../../libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/bin/../lib/libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/bin/../lib/libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/usr/local/lib/libgfortran.5.dylib' (no such file), '/usr/lib/libgfortran.5.dylib' (no such file, not in dyld cache)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/__init__.py", line 130, in <module>
    from numpy.__config__ import show as show_config
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/__config__.py", line 4, in <module>
    from numpy.core._multiarray_umath import (
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/__init__.py", line 50, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.11 from "/Users/victor/.micromamba/envs/nextstrain-ambient/bin/python3.11"
  * The NumPy version is: "1.26.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/_multiarray_umath.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libgfortran.5.dylib
  Referenced from: <CAA510D4-816A-34E5-9021-485EF5D56159> /Users/victor/.micromamba/envs/nextstrain-ambient/lib/libopenblas.0.dylib
  Reason: tried: '/Users/victor/.micromamba/envs/nextstrain-ambient/lib/libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/lib/libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/../../../../libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/core/../../../../libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/bin/../lib/libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/Users/victor/.micromamba/envs/nextstrain-ambient/bin/../lib/libgfortran.5.dylib' (duplicate LC_RPATH '@loader_path'), '/usr/local/lib/libgfortran.5.dylib' (no such file), '/usr/lib/libgfortran.5.dylib' (no such file, not in dyld cache)


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/bin/augur", line 6, in <module>
    from augur.__main__ import main
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/augur/__init__.py", line 12, in <module>
    from treetime import TreeTimeError, TreeTimeUnknownError
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/treetime/__init__.py", line 38, in <module>
    from .treeanc import TreeAnc
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/treetime/treeanc.py", line 3, in <module>
    import numpy as np
  File "/Users/victor/.micromamba/envs/nextstrain-ambient/lib/python3.11/site-packages/numpy/__init__.py", line 135, in <module>
    raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

Possible solutions

• Rebuild (blocked on Build fails on ubuntu-22.04 and macos-13 #105)
• Wait for upstream issue to be resolved: Numpy import error when using conda-forge build of numpy 2.2.3 // no issue with pip conda-forge/numpy-feedstock#347

[victorlin]

Maybe related (different numpy version but same error): conda-forge/numpy-feedstock#347

[genehack]

@victorlin making sure you saw the "duplicate RPATHs in macOS 15.4" stuff, and the linked shell script to de-duplicate them, at the very bottom of that conda-forge thread?

[victorlin]

Oh I missed that! I remember it was working fine on my work laptop which hasn't been updated. That makes sense now. So this is specific to macOS 15.4.

[genehack]

So this is specific to macOS 15.4.

FWIW, I just got a notification about the availability of macOS 15.4.1, if you want to roll the dice one more time…

[victorlin]

Can't on my work laptop (29 GB for an update is crazy), I'll try when I get home.

[victorlin]

Same error on 15.4.1.

[victorlin]

I'm using conda-base through Nextstrain CLI built with nextstrain/cli#436, running on macOS 15.4.1. The latest package works fine:

  conda 
    nextstrain-base 20250512T203714Z (0, https://conda.anaconda.org/nextstrain/osx-arm64)
    augur 30.0.1

I downgraded to confirm that it's still indeed broken with the older version:

  conda 
    nextstrain-base 20250311T191147Z (h60d57d3_0_locked, https://conda.anaconda.org/nextstrain/osx-arm64)
Traceback (most recent call last):
  File "/Users/victor/.nextstrain/runtimes/conda/env/lib/python3.11/site-packages/numpy/core/__init__.py", line 24, in <module>
    from . import multiarray
  File "/Users/victor/.nextstrain/runtimes/conda/env/lib/python3.11/site-packages/numpy/core/multiarray.py", line 10, in <module>
    from . import overrides
…

Not sure what changed, but this is no longer an issue.

[victorlin]

20250512T163842Z is the first working version. That means this was either resolved by changes from #111 or whatever happened with Conda packages between March 11 ~ May 12 (potentially too much to pin down specifics).

[jameshadfield]

For posterity, I also got this with a previously working (ambient/native) conda environment when MacOS updated to 15.5 (this is all arch64, I don't have rosetta installed). Destroying & rebuilding the conda environment (installing augur via pip install -e '.[dev] etc) gave me back a functional environment. Such is life.