Problem installing and using Oceananigans w Julia 1.7 + Apple M1 chip

[oseanography]

Hello, I am having some technical trouble installing and using Oceananigans. I am new to Julia and would like to try to set up Oceananigans on a Macbook. But it is running into issues.

(1) - installation process says some things are missing:

    Building MPI → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/d56a80d8cf8b9dc3050116346b3d83432b1912c0/build.log`
Precompiling project...
  ✗ MPI
  ✗ PencilArrays
  ✗ PencilFFTs
  ✗ Oceananigans
  89 dependencies successfully precompiled in 41 seconds
  4 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages

(2) - using Oceananigans is looking for MPI, but to my knowledge if I want to run on my computer, it shouldn't need MPI, like MITgcm using serial instead of parallel. But I don't know where to change the setting? This is what happens when I call using Oceananigans:

julia> using Oceananigans
[ Info: Precompiling Oceananigans [9e8cae18-63c1-5223-a75c-80ca9d6e9a09]
ERROR: LoadError: MPI.jl not properly configured, please run `Pkg.build("MPI")`.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] top-level scope
   @ ~/.julia/packages/MPI/08SPr/src/MPI.jl:38
 [3] include
   @ ./Base.jl:418 [inlined]
 [4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
   @ Base ./loading.jl:1318
 [5] top-level scope
   @ none:1
 [6] eval
   @ ./boot.jl:373 [inlined]
 [7] eval(x::Expr)
   @ Base.MainInclude ./client.jl:453
 [8] top-level scope
   @ none:1
in expression starting at /Users/sean/.julia/packages/MPI/08SPr/src/MPI.jl:1

caused by: LoadError: InitError: could not load library "/Users/sean/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/libmpifort.12.dylib"
dlopen(/Users/sean/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/libmpifort.12.dylib, 0x0001): Library not loaded: @rpath/libquadmath.0.dylib
  Referenced from: /Users/sean/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/libmpifort.12.dylib
  Reason: tried: '/Users/sean/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/./libquadmath.0.dylib' (no such file), '/Users/sean/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/./libquadmath.0.dylib' (no such file), '/Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libquadmath.0.dylib' (no such file), '/Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/libquadmath.0.dylib' (no such file), '/usr/local/lib/libquadmath.0.dylib' (no such file), '/usr/lib/libquadmath.0.dylib' (no such file)
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:117
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ MPICH_jll ~/.julia/packages/MPICH_jll/dhUyI/src/wrappers/aarch64-apple-darwin-libgfortran5.jl:32
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:768
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:854
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1097
  [8] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [9] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
 [10] top-level scope
    @ ~/.julia/packages/MPI/08SPr/deps/deps.jl:8
 [11] include(mod::Module, _path::String)
    @ Base ./Base.jl:418
 [12] include(x::String)
    @ MPI ~/.julia/packages/MPI/08SPr/src/MPI.jl:1
 [13] top-level scope
    @ ~/.julia/packages/MPI/08SPr/src/MPI.jl:36
 [14] include
    @ ./Base.jl:418 [inlined]
 [15] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1318
 [16] top-level scope
    @ none:1
 [17] eval
    @ ./boot.jl:373 [inlined]
 [18] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [19] top-level scope
    @ none:1
during initialization of module MPICH_jll
in expression starting at /Users/sean/.julia/packages/MPI/08SPr/deps/deps.jl:1
ERROR: LoadError: Failed to precompile MPI [da04e1cc-30fd-572f-bb4f-1f8673147195] to /Users/sean/.julia/compiled/v1.7/MPI/jl_AfEwik.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1466
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1410
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1120
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
  [7] include(mod::Module, _path::String)
    @ Base ./Base.jl:418
  [8] include(x::String)
    @ Oceananigans ~/.julia/packages/Oceananigans/jmNfq/src/Oceananigans.jl:5
  [9] top-level scope
    @ ~/.julia/packages/Oceananigans/jmNfq/src/Oceananigans.jl:190
 [10] include
    @ ./Base.jl:418 [inlined]
 [11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1318
 [12] top-level scope
    @ none:1
 [13] eval
    @ ./boot.jl:373 [inlined]
 [14] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [15] top-level scope
    @ none:1
in expression starting at /Users/sean/.julia/packages/Oceananigans/jmNfq/src/Distributed/Distributed.jl:1
in expression starting at /Users/sean/.julia/packages/Oceananigans/jmNfq/src/Oceananigans.jl:1
ERROR: Failed to precompile Oceananigans [9e8cae18-63c1-5223-a75c-80ca9d6e9a09] to /Users/sean/.julia/compiled/v1.7/Oceananigans/jl_sx8KhM.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1466
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1410
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1120
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997

Any help would be greatly appreciated. Thanks

[navidcy]

I'm moving this to a discussion and we can continue there. :)

[navidcy]

What Julia version you have? E.g., can you print out the output of

julia> versioninfo()

?

[oseanography]

I have 1.7 -

julia> versioninfo()
Julia Version 1.7.2
Commit bf53498635 (2022-02-06 15:21 UTC)
Platform Info:
OS: macOS (arm64-apple-darwin21.2.0)
CPU: Apple M1 Max
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-12.0.1 (ORCJIT, cyclone)

[navidcy]

Same problem here with Julia 1.7 on MacBook Pro M1...

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.2 (2022-02-06)
 _/ |\__'_|_|_|\__'_|  |
|__/                   |

(@v1.7) pkg> activate .
  Activating new project at `~/Research/test_Oceananigans`

(test_Oceananigans) pkg> add Oceananigans
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
   Installed Static ─────── v0.6.2
   Installed Parsers ────── v2.3.1
   Installed NCDatasets ─── v0.12.4
   Installed Oceananigans ─ v0.75.2
    Updating `~/Research/test_Oceananigans/Project.toml`
  [9e8cae18] + Oceananigans v0.75.2
    Updating `~/Research/test_Oceananigans/Manifest.toml`
  [621f4979] + AbstractFFTs v1.1.0
  [79e6a3ab] + Adapt v3.3.3
  [4fba245c] + ArrayInterface v5.0.7
  [ab4f0b2a] + BFloat16s v0.1.0
  [fa961155] + CEnum v0.4.2
  [179af706] + CFTime v0.1.2
  [052768ef] + CUDA v3.3.6
  [72cfdca4] + CUDAKernels v0.3.0
  [49dc2e85] + Calculus v0.5.1
  [7057c7e9] + Cassette v0.3.10
  [d360d2e6] + ChainRulesCore v1.14.0
  [9e997f8a] + ChangesOfVariables v0.1.2
  [34da2185] + Compat v3.43.0
  [a8cc5b0e] + Crayons v4.1.1
  [7445602f] + CubedSphere v0.2.0
  [9a962f9c] + DataAPI v1.10.0
  [864edb3b] + DataStructures v0.18.11
  [e2d170a0] + DataValueInterfaces v1.0.0
  [b552c78f] + DiffRules v1.10.0
  [ffbed154] + DocStringExtensions v0.8.6
  [fa6b7ba4] + DualNumbers v0.6.8
  [b305315f] + Elliptic v1.0.1
  [e2ba6199] + ExprTools v0.1.8
  [7a1cc6ca] + FFTW v1.4.6
  [5789e2e9] + FileIO v1.13.0
  [0c68f7d7] + GPUArrays v8.3.2
  [61eb1bfa] + GPUCompiler v0.12.9
  [c27321d9] + Glob v1.3.0
  [615f187c] + IfElse v0.1.1
  [40713840] + IncompleteLU v0.2.0
  [3587e190] + InverseFunctions v0.1.3
  [92d709cd] + IrrationalConstants v0.1.1
  [42fd0dbc] + IterativeSolvers v0.9.2
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [033835bb] + JLD2 v0.4.22
  [692b3bcd] + JLLWrappers v1.4.1
  [0f8b85d8] + JSON3 v1.9.4
  [63c18a36] + KernelAbstractions v0.7.2
  [929cbde3] + LLVM v4.9.1
  [2ab3a3ac] + LogExpFunctions v0.3.12
  [da04e1cc] + MPI v0.19.2
  [1914dd2f] + MacroTools v0.5.9
  [85f8d34a] + NCDatasets v0.12.4
  [77ba4419] + NaNMath v1.0.0
  [9e8cae18] + Oceananigans v0.75.2
  [6fe1bfb0] + OffsetArrays v1.10.8
  [bac558e1] + OrderedCollections v1.4.1
  [69de0a69] + Parsers v2.3.1
  [0e08944d] + PencilArrays v0.16.1
  [4a48f351] + PencilFFTs v0.13.4
  [21216c6a] + Preferences v1.3.0
  [94ee1d12] + Quaternions v0.5.6
  [74087812] + Random123 v1.5.0
  [e6cf234a] + RandomNumbers v1.5.3
  [3cdcf5f2] + RecipesBase v1.2.1
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.0
  [6038ab10] + Rotations v1.3.1
  [1bc83da4] + SafeTestsets v0.0.1
  [d496a93d] + SeawaterPolynomials v0.2.2
  [276daf66] + SpecialFunctions v1.8.4
  [aedffcd0] + Static v0.6.2
  [90137ffa] + StaticArrays v1.4.4
  [15972242] + StaticPermutations v0.3.0
  [5e0ebb24] + Strided v1.2.1
  [09ab397b] + StructArrays v0.6.5
  [856f2bd8] + StructTypes v1.8.1
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.7.0
  [6aa5eb33] + TaylorSeries v0.10.13
  [a759f4b9] + TimerOutputs v0.5.17
  [3bb67fe8] + TranscodingStreams v0.9.6
  [bc48ee85] + Tullio v0.3.4
  [9d95972d] + TupleTools v1.3.0
  [81def892] + VersionParsing v1.3.0
  [f5851436] + FFTW_jll v3.3.10+0
  [0234f1f7] + HDF5_jll v1.12.0+1
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+2
  [dad2f222] + LLVMExtra_jll v0.0.14+2
  [856f044c] + MKL_jll v2022.0.0+0
  [7cb0a576] + MPICH_jll v4.0.2+0
  [9237b28f] + MicrosoftMPI_jll v10.1.3+2
  [7243133f] + NetCDF_jll v400.702.400+0
  [fe0851c0] + OpenMPI_jll v4.1.2+0
  [458c3c95] + OpenSSL_jll v1.1.14+0
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [f43a241f] + Downloads
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  [c8ffd9c3] + MbedTLS_jll
  [14a3606d] + MozillaCACerts_jll
  [4536629a] + OpenBLAS_jll
  [05823500] + OpenLibm_jll
  [83775a58] + Zlib_jll
  [8e850b90] + libblastrampoline_jll
  [8e850ede] + nghttp2_jll
  [3f19e933] + p7zip_jll
Precompiling project...
  ✗ MPI
  ✗ PencilArrays
  ✗ PencilFFTs
  ✗ Oceananigans
  17 dependencies successfully precompiled in 34 seconds (72 already precompiled)
  4 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages

[navidcy]

But all seems OK on Julia 1.6

(base) navid:test_Oceananigans/ $ julia-1.6                          [10:11:18]
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.6 (2022-03-28)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.6) pkg> activate .
  Activating environment at `~/Research/test_Oceananigans/Project.toml`

(test_Oceananigans) pkg> add Oceananigans
    Updating registry at `~/.julia/registries/General`
   Resolving package versions...
    Updating `~/Research/test_Oceananigans/Project.toml`
  [9e8cae18] + Oceananigans v0.75.2
    Updating `~/Research/test_Oceananigans/Manifest.toml`
  [621f4979] + AbstractFFTs v1.1.0
  [79e6a3ab] + Adapt v3.3.3
  [4fba245c] + ArrayInterface v5.0.7
  [ab4f0b2a] + BFloat16s v0.1.0
  [fa961155] + CEnum v0.4.2
  [179af706] + CFTime v0.1.2
  [052768ef] + CUDA v3.3.6
  [72cfdca4] + CUDAKernels v0.3.0
  [49dc2e85] + Calculus v0.5.1
  [7057c7e9] + Cassette v0.3.10
  [d360d2e6] + ChainRulesCore v1.14.0
  [9e997f8a] + ChangesOfVariables v0.1.2
  [34da2185] + Compat v3.43.0
  [a8cc5b0e] + Crayons v4.1.1
  [7445602f] + CubedSphere v0.2.0
  [9a962f9c] + DataAPI v1.10.0
  [864edb3b] + DataStructures v0.18.11
  [e2d170a0] + DataValueInterfaces v1.0.0
  [b552c78f] + DiffRules v1.10.0
  [ffbed154] + DocStringExtensions v0.8.6
  [fa6b7ba4] + DualNumbers v0.6.8
  [b305315f] + Elliptic v1.0.1
  [e2ba6199] + ExprTools v0.1.8
  [7a1cc6ca] + FFTW v1.4.6
  [5789e2e9] + FileIO v1.13.0
  [0c68f7d7] + GPUArrays v8.3.2
  [61eb1bfa] + GPUCompiler v0.12.9
  [c27321d9] + Glob v1.3.0
  [615f187c] + IfElse v0.1.1
  [40713840] + IncompleteLU v0.2.0
  [3587e190] + InverseFunctions v0.1.3
  [92d709cd] + IrrationalConstants v0.1.1
  [42fd0dbc] + IterativeSolvers v0.9.2
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [033835bb] + JLD2 v0.4.22
  [692b3bcd] + JLLWrappers v1.4.1
  [0f8b85d8] + JSON3 v1.9.4
  [63c18a36] + KernelAbstractions v0.7.2
  [929cbde3] + LLVM v4.9.1
  [2ab3a3ac] + LogExpFunctions v0.3.12
  [da04e1cc] + MPI v0.19.2
  [1914dd2f] + MacroTools v0.5.9
  [85f8d34a] + NCDatasets v0.12.4
  [77ba4419] + NaNMath v1.0.0
  [9e8cae18] + Oceananigans v0.75.2
  [6fe1bfb0] + OffsetArrays v1.10.8
  [bac558e1] + OrderedCollections v1.4.1
  [69de0a69] + Parsers v2.3.1
  [0e08944d] + PencilArrays v0.16.1
  [4a48f351] + PencilFFTs v0.13.4
  [21216c6a] + Preferences v1.3.0
  [94ee1d12] + Quaternions v0.5.6
  [74087812] + Random123 v1.5.0
  [e6cf234a] + RandomNumbers v1.5.3
  [3cdcf5f2] + RecipesBase v1.2.1
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.0
  [6038ab10] + Rotations v1.3.1
  [1bc83da4] + SafeTestsets v0.0.1
  [d496a93d] + SeawaterPolynomials v0.2.2
  [276daf66] + SpecialFunctions v1.8.4
  [aedffcd0] + Static v0.6.2
  [90137ffa] + StaticArrays v1.4.4
  [15972242] + StaticPermutations v0.3.0
  [5e0ebb24] + Strided v1.2.1
  [09ab397b] + StructArrays v0.6.5
  [856f2bd8] + StructTypes v1.8.1
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.7.0
  [6aa5eb33] + TaylorSeries v0.10.13
  [a759f4b9] + TimerOutputs v0.5.17
  [3bb67fe8] + TranscodingStreams v0.9.6
  [bc48ee85] + Tullio v0.3.4
  [9d95972d] + TupleTools v1.3.0
  [81def892] + VersionParsing v1.3.0
  [f5851436] + FFTW_jll v3.3.10+0
  [0234f1f7] + HDF5_jll v1.12.0+1
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+2
  [dad2f222] + LLVMExtra_jll v0.0.14+2
  [856f044c] + MKL_jll v2022.0.0+0
  [7cb0a576] + MPICH_jll v4.0.2+0
  [9237b28f] + MicrosoftMPI_jll v10.1.3+2
  [7243133f] + NetCDF_jll v400.702.400+0
  [fe0851c0] + OpenMPI_jll v4.1.2+0
  [458c3c95] + OpenSSL_jll v1.1.14+0
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [f43a241f] + Downloads
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  [c8ffd9c3] + MbedTLS_jll
  [14a3606d] + MozillaCACerts_jll
  [05823500] + OpenLibm_jll
  [83775a58] + Zlib_jll
  [8e850ede] + nghttp2_jll
  [3f19e933] + p7zip_jll
Precompiling project...
  34 dependencies successfully precompiled in 49 seconds (57 already precompiled)

[navidcy]

on Julia 1.7, I can't even install MPI by itself...

(base) navid:test_Oceananigans/ $ julia                                                                  [10:17:45]
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.2 (2022-02-06)
 _/ |\__'_|_|_|\__'_|  |
|__/                   |

(@v1.7) pkg> activate .
  Activating project at `~/Research/test_Oceananigans`

(test_Oceananigans) pkg> add MPI
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/Research/test_Oceananigans/Project.toml`
  [da04e1cc] + MPI v0.19.2
    Updating `~/Research/test_Oceananigans/Manifest.toml`
  [ffbed154] + DocStringExtensions v0.8.6
  [692b3bcd] + JLLWrappers v1.4.1
  [da04e1cc] + MPI v0.19.2
  [21216c6a] + Preferences v1.3.0
  [ae029012] + Requires v1.3.0
  [7cb0a576] + MPICH_jll v4.0.2+0
  [9237b28f] + MicrosoftMPI_jll v10.1.3+2
  [fe0851c0] + OpenMPI_jll v4.1.2+0
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8ba89e20] + Distributed
  [f43a241f] + Downloads
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  [c8ffd9c3] + MbedTLS_jll
  [14a3606d] + MozillaCACerts_jll
  [83775a58] + Zlib_jll
  [8e850ede] + nghttp2_jll
  [3f19e933] + p7zip_jll

(test_Oceananigans) pkg> build MPI
    Building MPI → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/d56a80d8cf8b9dc3050116346b3d83432b1912c0/build.log`

julia> using MPI
ERROR: InitError: could not load library "/Users/navid/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/libmpifort.12.dylib"
dlopen(/Users/navid/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/libmpifort.12.dylib, 0x0001): Library not loaded: @rpath/libquadmath.0.dylib
  Referenced from: /Users/navid/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/libmpifort.12.dylib
  Reason: tried: '/Users/navid/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/./libquadmath.0.dylib' (no such file), '/Users/navid/.julia/artifacts/48a9a608db31268626d8b8d4d1272c3e7ccbf7d5/lib/./libquadmath.0.dylib' (no such file), '/Users/navid/julia-1.7/usr/lib/libquadmath.0.dylib' (no such file), '/Users/navid/julia-1.7/usr/bin/../lib/libquadmath.0.dylib' (no such file), '/usr/local/lib/libquadmath.0.dylib' (no such file), '/usr/lib/libquadmath.0.dylib' (no such file)
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:117
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ MPICH_jll ~/.julia/packages/MPICH_jll/dhUyI/src/wrappers/aarch64-apple-darwin-libgfortran5.jl:32
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:768
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:854
  [7] _tryrequire_from_serialized(modkey::Base.PkgId, build_id::UInt64, modpath::String)
    @ Base ./loading.jl:783
  [8] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:843
  [9] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1097
 [10] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
 [11] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
during initialization of module MPICH_jll

[navidcy]

I think we should raise this issue at MPI.jl?

[navidcy]

JuliaParallel/MPI.jl#583

[glwagner]

@oseanography first of all amazing github handle.

You might be able to fix this problem by first adding MPICH_jll at version 4.0.1 (we just found this bug). One liner:

julia> using Pkg; Pkg.add(name="MPICH_jll", version="4.0.1"); Pkg.build()

then restart Julia and try again to add Oceananigans, etc.

[navidcy]

Yeap, that works!

(base) navid:test_Oceananigans/ $ julia --project                                                        [10:52:37]
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.2 (2022-02-06)
 _/ |\__'_|_|_|\__'_|  |
|__/                   |

julia> using Pkg; Pkg.add(name="MPICH_jll", version="4.0.1"); Pkg.build()
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/Research/test_Oceananigans/Project.toml`
  [7cb0a576] + MPICH_jll v4.0.1+0
    Updating `~/Research/test_Oceananigans/Manifest.toml`
  [692b3bcd] + JLLWrappers v1.4.1
  [21216c6a] + Preferences v1.3.0
  [7cb0a576] + MPICH_jll v4.0.1+0
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  [c8ffd9c3] + MbedTLS_jll
  [14a3606d] + MozillaCACerts_jll
  [83775a58] + Zlib_jll
  [8e850ede] + nghttp2_jll
  [3f19e933] + p7zip_jll

(test_Oceananigans) pkg> add Oceananigans
   Resolving package versions...
    Updating `~/Research/test_Oceananigans/Project.toml`
  [9e8cae18] + Oceananigans v0.75.2
    Updating `~/Research/test_Oceananigans/Manifest.toml`
  [621f4979] + AbstractFFTs v1.1.0
  [79e6a3ab] + Adapt v3.3.3
  [4fba245c] + ArrayInterface v5.0.7
  [ab4f0b2a] + BFloat16s v0.1.0
  [fa961155] + CEnum v0.4.2
  [179af706] + CFTime v0.1.2
  [052768ef] + CUDA v3.3.6
  [72cfdca4] + CUDAKernels v0.3.0
  [49dc2e85] + Calculus v0.5.1
  [7057c7e9] + Cassette v0.3.10
  [d360d2e6] + ChainRulesCore v1.14.0
  [9e997f8a] + ChangesOfVariables v0.1.2
  [34da2185] + Compat v3.43.0
  [a8cc5b0e] + Crayons v4.1.1
  [7445602f] + CubedSphere v0.2.0
  [9a962f9c] + DataAPI v1.10.0
  [864edb3b] + DataStructures v0.18.11
  [e2d170a0] + DataValueInterfaces v1.0.0
  [b552c78f] + DiffRules v1.10.0
  [ffbed154] + DocStringExtensions v0.8.6
  [fa6b7ba4] + DualNumbers v0.6.8
  [b305315f] + Elliptic v1.0.1
  [e2ba6199] + ExprTools v0.1.8
  [7a1cc6ca] + FFTW v1.4.6
  [5789e2e9] + FileIO v1.13.0
  [0c68f7d7] + GPUArrays v8.3.2
  [61eb1bfa] + GPUCompiler v0.12.9
  [c27321d9] + Glob v1.3.0
  [615f187c] + IfElse v0.1.1
  [40713840] + IncompleteLU v0.2.0
  [3587e190] + InverseFunctions v0.1.3
  [92d709cd] + IrrationalConstants v0.1.1
  [42fd0dbc] + IterativeSolvers v0.9.2
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [033835bb] + JLD2 v0.4.22
  [0f8b85d8] + JSON3 v1.9.4
  [63c18a36] + KernelAbstractions v0.7.2
  [929cbde3] + LLVM v4.9.1
  [2ab3a3ac] + LogExpFunctions v0.3.12
  [da04e1cc] + MPI v0.19.2
  [1914dd2f] + MacroTools v0.5.9
  [85f8d34a] + NCDatasets v0.12.4
  [77ba4419] + NaNMath v1.0.0
  [9e8cae18] + Oceananigans v0.75.2
  [6fe1bfb0] + OffsetArrays v1.10.8
  [bac558e1] + OrderedCollections v1.4.1
  [69de0a69] + Parsers v2.3.1
  [0e08944d] + PencilArrays v0.16.1
  [4a48f351] + PencilFFTs v0.13.4
  [94ee1d12] + Quaternions v0.5.6
  [74087812] + Random123 v1.5.0
  [e6cf234a] + RandomNumbers v1.5.3
  [3cdcf5f2] + RecipesBase v1.2.1
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.0
  [6038ab10] + Rotations v1.3.1
  [1bc83da4] + SafeTestsets v0.0.1
  [d496a93d] + SeawaterPolynomials v0.2.2
  [276daf66] + SpecialFunctions v1.8.4
  [aedffcd0] + Static v0.6.2
  [90137ffa] + StaticArrays v1.4.4
  [15972242] + StaticPermutations v0.3.0
  [5e0ebb24] + Strided v1.2.1
  [09ab397b] + StructArrays v0.6.5
  [856f2bd8] + StructTypes v1.8.1
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.7.0
  [6aa5eb33] + TaylorSeries v0.10.13
  [a759f4b9] + TimerOutputs v0.5.17
  [3bb67fe8] + TranscodingStreams v0.9.6
  [bc48ee85] + Tullio v0.3.4
  [9d95972d] + TupleTools v1.3.0
  [81def892] + VersionParsing v1.3.0
  [f5851436] + FFTW_jll v3.3.10+0
  [0234f1f7] + HDF5_jll v1.12.0+1
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+2
  [dad2f222] + LLVMExtra_jll v0.0.14+2
  [856f044c] + MKL_jll v2022.0.0+0
  [9237b28f] + MicrosoftMPI_jll v10.1.3+2
  [7243133f] + NetCDF_jll v400.702.400+0
  [fe0851c0] + OpenMPI_jll v4.1.2+0
  [458c3c95] + OpenSSL_jll v1.1.14+0
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [4af54fe1] + LazyArtifacts
  [37e2e46d] + LinearAlgebra
  [a63ad114] + Mmap
  [1a1011a3] + SharedArrays
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [8dfed614] + Test
  [4536629a] + OpenBLAS_jll
  [05823500] + OpenLibm_jll
  [8e850b90] + libblastrampoline_jll
Precompiling project...
  3 dependencies successfully precompiled in 17 seconds (90 already precompiled)

julia> using Oceananigans
[ Info: Precompiling Oceananigans [9e8cae18-63c1-5223-a75c-80ca9d6e9a09]
WARNING: Method definition diffusive_flux_z(Any, Any, Any, Any, Oceananigans.TurbulenceClosures.AbstractScalarDiffusivity{var"#s139", var"#s138"} where var"#s138"<:Oceananigans.TurbulenceClosures.HorizontalDivergenceFormulation where var"#s139", Any, Base.Val{x} where x, Any...) in module TurbulenceClosures at /Users/navid/.julia/packages/Oceananigans/MVUJA/src/TurbulenceClosures/abstract_scalar_diffusivity_closure.jl:192 overwritten at /Users/navid/.julia/packages/Oceananigans/MVUJA/src/TurbulenceClosures/abstract_scalar_diffusivity_closure.jl:204.
  ** incremental compilation may be fatally broken for this module **

[ Info: Oceananigans will use 6 threads

(the warning about WARNING: Method definition diffusive_flux_z... is fixed in Oceananigans v0.75.3...)

[glwagner]

Thanks @oseanography for finding this !!

[navidcy]

So, Oceananigans v0.75.3 will resolve this and one need not be doing julia> using Pkg; Pkg.add(name="MPICH_jll", version="4.0.1"); Pkg.build(), etc...