Segfault on x86_64 platforms related to passing `Vector` through `ccall`

[BioTurboNick]

From here: JuliaLang/LinearAlgebra.jl#1190

There appears to be a platform-specific issue where, in certain cases, initiating a vector and then resizing or reshaping it can result in a segfault (Linux) or EXCEPTION_ACCESS_VIOLATION or ReadOnlyMemoryError (Windows). Error is sometimes sporadic.

Only seems to affect x86_64 and (at least) Julia 1.9 through nightly. Does not seem to affect Linux i686 or MacOS-aarch64 (on at least nightly).

Broken (in the larger context of the stegr! method):

# A::Matrix{T <: AbstractFloat}
Z = similar(A, T, n * m)
# ...
return reshape(resize!(Z, n * m), n, m)

Works:

# A::Matrix{T <: AbstractFloat}
Z = similar(A, T, n, m)
# ...
return reshape(resize!(vec(Z), n * m), n, m)

Can execute this on nightly to reproduce:

        const elty = Float64
        using LinearAlgebra
        import LinearAlgebra:LAPACK
        import Base: require_one_based_indexing
        import LinearAlgebra.LAPACK: @chkvalidparam, chklapackerror, @blasfunc
        import LinearAlgebra: chkstride1, BlasInt, libblastrampoline
        function LAPACK.stegr!(jobz::AbstractChar, range::AbstractChar, dv::AbstractVector{elty}, ev::AbstractVector{elty}, vl::Real, vu::Real, il::Integer, iu::Integer)
            require_one_based_indexing(dv, ev)
            chkstride1(dv, ev)
            n = length(dv)
            ne = length(ev)
            if ne == n - 1
                eev = [ev; zero(elty)]
            elseif ne == n
                eev = copy(ev)
                eev[n] = zero(elty)
            else
                throw(DimensionMismatch(lazy"ev has length $ne but needs one less than or equal to dv's length, $n)"))
            end

            abstol = Vector{elty}(undef, 1)
            m = Ref{BlasInt}()
            w = similar(dv, elty, n)
            ldz = jobz == 'N' ? 1 : n
            Zn = range == 'I' ? iu-il+1 : n
            Z = similar(dv, elty, ldz * Zn)
            isuppz = similar(dv, BlasInt, 2*size(Z, 2))
            work = Vector{elty}(undef, 1)
            lwork = BlasInt(-1)
            iwork = Vector{BlasInt}(undef, 1)
            liwork = BlasInt(-1)
            info = Ref{BlasInt}()
            for i = 1:2  # first call returns lwork as work[1] and liwork as iwork[1]
                ccall((@blasfunc(dstegr_), libblastrampoline), Cvoid,
                    (Ref{UInt8}, Ref{UInt8}, Ref{BlasInt}, Ptr{elty},
                    Ptr{elty}, Ref{elty}, Ref{elty}, Ref{BlasInt},
                    Ref{BlasInt}, Ptr{elty}, Ptr{BlasInt}, Ptr{elty},
                    Ptr{elty}, Ref{BlasInt}, Ptr{BlasInt}, Ptr{elty},
                    Ref{BlasInt}, Ptr{BlasInt}, Ref{BlasInt}, Ptr{BlasInt},
                    Clong, Clong),
                    jobz, range, n, dv,
                    eev, vl, vu, il,
                    iu, abstol, m, w,
                    Z, ldz, isuppz, work,
                    lwork, iwork, liwork, info,
                    1, 1)
                chklapackerror(info[])
                if i == 1
                    lwork = BlasInt(work[1])
                    resize!(work, lwork)
                    liwork = iwork[1]
                    resize!(iwork, liwork)
                end
            end
            wm = min(n, m[])
            Zm = min(Zn, m[])
            resize!(w, wm), reshape(resize!(Z, ldz * Zm), ldz, Zm)
        end
a = SymTridiagonal([1.0129540363709792, 0.5125544740879742, 1.4404024510136622, 0.9645500736079791, 0.8349894041911953, 1.0404785847552251, 0.9418430974856163, 0.1401583600348122, 0.38426770335791244, 0.4551768643291618, 0.4200328536585094, 1.8593998829530265e-7], [0.40739613651353346, 0.10643163002740841, 0.6222058356713083, 0.33352809216796453, 0.3799591152214755, 0.21073828255014115, 0.8158852900425236, 0.1683998539581063, 0.5002702262460241, 0.6123852845929411, 0.7701699043551441])
A = copy(a); LAPACK.stegr!('V', 'A', A.dv, A.ev, 0.0, 0.0, 0, 0)

[BioTurboNick]

Generated rr trace: https://julialang-dumps.s3.amazonaws.com/reports/2025-02-12T21-24-29-BioTurboNick.tar.zst

[BioTurboNick]

Poking around more, the resize/reshape seems to be dispensable.

It seems to be that a Vector passed into the ccall ends up corrupting something, whereas a Matrix of the same type and element count does not. At least one other LAPACK method doesn't seem to have an issue with this.

If I run the function line by line in the REPL, simply typing a character on the REPL after the second ccall may crash Julia.

[gbaraldi]

Is the ccall being done correctly? This looks like the blas call is corrupting julia memory

[BioTurboNick]

As far as I can tell it is fine? At least, I didn't modify it from what it has been for years.

As I'm looking up information on this, maybe there's a subtlety about LAPACK being a FORTRAN library that I'm missing? For instance, I see that FORTRAN is stricter about array dimensionality than C; is that encoded in the call somehow?

If I create it as a vector, and then pass in as a matrix via reshape, it still crashes. And if I create it as a matrix and then reshape it to a vector before passing, it doesn't crash. Extending this observation, if I create it as a 3d array with a 1-trailing dimension, no crash. If I create it as a 3d array with a 1-length dimension in the middle, crash.

But cconvert just pulls out the MemoryRef, which seems to be identical (apart from the pointer location) no matter whether the source was a matrix or a vector. So then where would the dependence on the initial shape come in?

[BioTurboNick]

Oh. Nevermind - stupid error. 😅 (if you change to a vector, don't rely on the size of it's second dimension...)