Skip to content

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Feb 6, 2025

This pull request changes the compat entry for the LinearSolve package from 2.21 to 2.21, 3.
This keeps the compat entries for earlier versions.

Note: I have not tested your package with this new compat entry.
It is your responsibility to make sure that your package tests pass before you merge this pull request.

Closes #135, closes #136, closes #137

@SKopecz SKopecz force-pushed the compathelper/new_version/2025-02-06-01-27-57-156-03022304862 branch from dcd2142 to 76c49f7 Compare February 6, 2025 01:28
@codecov-commenter
Copy link

codecov-commenter commented Feb 6, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.26%. Comparing base (4a2db1a) to head (76c49f7).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #131   +/-   ##
=======================================
  Coverage   98.26%   98.26%           
=======================================
  Files           7        7           
  Lines        1556     1556           
=======================================
  Hits         1529     1529           
  Misses         27       27           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions bot and others added 3 commits February 6, 2025 09:09
…p existing compat) (#132)

Co-authored-by: CompatHelper Julia <[email protected]>
…(keep existing compat) (#133)

Co-authored-by: CompatHelper Julia <[email protected]>
…p existing compat) (#134)

Co-authored-by: CompatHelper Julia <[email protected]>
Copy link
Member

@ranocha ranocha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the release notes of LinearSolve.jl

RecrusiveFactorization.jl removed as a dependency and turned into an extension. It now must be explicitly loaded (using RecrusiveFactorization) in order to be used. It's still a part of the default algorithm but will only be selected if loaded. The reason for this is because this method brings in the LoopVectorization.jl stack and is thus a heavy dependency that can invalidate a large amount of code. This greatly reduces first run and solve times downstream. However, as it is the fastest method for many scenarios, it is still recommended that many users opt-in if they are looking for performance, but we see this makes a better trade-off between default performance and first run times.

See https://github.com/SciML/LinearSolve.jl/releases/tag/v3.0.0

I think we use RecursiveFactorization.jl as default, so we have to adapt this

@ranocha
Copy link
Member

ranocha commented Feb 17, 2025

We should update to Julia version 1.10 - also required for #121

@ranocha
Copy link
Member

ranocha commented Feb 22, 2025

There are some errors, e.g.,

  Got exception outside of a @test
  ArgumentError: pattern of the matrix changed
  Stacktrace:
    [1] umferror(status::Int32)
      @ SparseArrays.UMFPACK ~/hostedtoolcache/julia/1.10.8/aarch64/share/julia/stdlib/v1.10/SparseArrays/src/solvers/umfpack.jl:123

https://github.com/SKopecz/PositiveIntegrators.jl/actions/runs/13472498519/job/37647716517?pr=131#step:6:950

@JoshuaLampert
Copy link
Member

There are some errors, e.g.,

  Got exception outside of a @test
  ArgumentError: pattern of the matrix changed
  Stacktrace:
    [1] umferror(status::Int32)
      @ SparseArrays.UMFPACK ~/hostedtoolcache/julia/1.10.8/aarch64/share/julia/stdlib/v1.10/SparseArrays/src/solvers/umfpack.jl:123

SKopecz/PositiveIntegrators.jl/actions/runs/13472498519/job/37647716517?pr=131#step:6:950

I tried to debug this error, but really don't understand why we get the error. This is a reduced version:

using LinearAlgebra
using SparseArrays
using PositiveIntegrators

prod! = (P, u, p, t) -> begin
    fill!(P, zero(eltype(P)))
    # for j in axes(P, 2)
    #     for idx in nzrange(P, j)
    #         i = rowvals(P)[idx]
    #         nonzeros(P)[idx] = i * u[i]
    #     end
    # end
    for i in 1:(length(u) - 1)
        P[i, i + 1] = i * u[i]
    end
    return nothing
end

n = 4
P_tridiagonal = Tridiagonal([0.1, 0.2, 0.3],
                            zeros(n),
                            [0.4, 0.5, 0.6])
P_sparse = sparse(P_tridiagonal)
u0 = [1.0, 1.5, 2.0, 2.5]
tspan = (0.0, 1.0)
dt = 0.25

alg = MPE()
display(P_sparse)
prob_sparse_ip = ConservativePDSProblem(prod!, u0, tspan;
                                        p_prototype = P_sparse)
sol_sparse_ip = solve(prob_sparse_ip, alg; dt, adaptive = false)

This errors with LinearSolve.jl v3, but doesn't with LinearSolve.jl v2.39.0. As you can see in the commented out code above, I also tried the version of setting the sparse production matrix recommended in https://skopecz.github.io/PositiveIntegrators.jl/stable/faq/, but this didn't help. I also double-checked that the sparsity pattern is the same before calling solve! from LinearSolve.jl here:
https://github.com/SKopecz/PositiveIntegrators.jl/blob/79e83680ed86d78fdbfcf95b6808a99062285b43/src/mprk.jl#L433
as it is in the p_prototype in the beginning, which is the case. I'm not sure what else to try. Any ideas?

@JoshuaLampert
Copy link
Member

With the recent bandaid, allowing LinearSolve.jl v3 gained importance. Do you have any idea what I could try out to debug the problem I described above, @ranocha ? Or do you see any other solution?

@ranocha
Copy link
Member

ranocha commented Mar 24, 2025

I don't really have a good idea.

  • While looking at the diff of LinearSolve.jl, I noticed that the aliasing arguments changed. Maybe we can try setting the aliasing of the matrix to false?
  • Are the versions of the underlying sparse solver libraries the same?
  • Does it only happen for the UMFPACK factorization or also for the KLUFactorization?

@JoshuaLampert
Copy link
Member

I don't really have a good idea.

  • While looking at the diff of LinearSolve.jl, I noticed that the aliasing arguments changed. Maybe we can try setting the aliasing of the matrix to false?
  • Are the versions of the underlying sparse solver libraries the same?
  • Does it only happen for the UMFPACK factorization or also for the KLUFactorization?

These are very good suggestions. Thanks!

@ranocha
Copy link
Member

ranocha commented Mar 24, 2025

Did you also try setting it via their new aliasing interface (like alias = LinearAliasSpecifier(alias_A = false, alias_b = true))?

  • I will check that.

👍

  • Using KLUFactorization() instead of LUFactorization() indeed fixes the problem.

Note that KLUFactorization() should only be used for sparse matrices.

@JoshuaLampert
Copy link
Member

Did you also try setting it via their new aliasing interface (like alias = LinearAliasSpecifier(alias_A = false, alias_b = true))?

Yes, this alone doesn't help (but is needed to fix the warnings).

@JoshuaLampert
Copy link
Member

JoshuaLampert commented Mar 24, 2025

Are the versions of the underlying sparse solver libraries the same?

Both LinearSolve.jl v2.39 (working) and LinearSolve.jl v3.7 (not working) use SuiteSparse_jll.jl v7.7.0+0. For completeness, here are the full versions:

LinearSolve.jl v2.39
(PositiveIntegrators) pkg> st -m
Project PositiveIntegrators v0.2.9-DEV
Status `~/.julia/dev/PositiveIntegrators/Manifest.toml`
  [47edcb42] ADTypes v1.14.0
  [7d9f7c33] Accessors v0.1.42
  [79e6a3ab] Adapt v4.3.0
  [4fba245c] ArrayInterface v7.18.0
  [4c555306] ArrayLayouts v1.11.1
  [62783981] BitTwiddlingConvenienceFunctions v0.1.6
  [2a0fbf3d] CPUSummary v0.2.6
  [d360d2e6] ChainRulesCore v1.25.1
  [fb6a15b2] CloseOpenIntervals v0.1.13
  [38540f10] CommonSolve v0.2.4
  [f70d9fcc] CommonWorldInvalidations v1.0.0
  [34da2185] Compat v4.16.0
  [a33af91c] CompositionsBase v0.1.2
  [2569d6c7] ConcreteStructs v0.2.3
  [187b0558] ConstructionBase v1.5.8
  [adafc99b] CpuId v0.3.1
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.22
  [e2d170a0] DataValueInterfaces v1.0.0
  [2b5f629d] DiffEqBase v6.167.0
  [ffbed154] DocStringExtensions v0.9.3
  [4e289a0a] EnumX v1.0.4
  [f151be2c] EnzymeCore v0.8.8
  [e2ba6199] ExprTools v0.1.10
  [55351af7] ExproniconLite v0.10.14
  [7034ab61] FastBroadcast v0.3.5
  [9aa1b823] FastClosures v0.3.2
  [29a986be] FastLapackInterface v2.0.4
  [a4df4552] FastPower v1.1.1
  [1a297f60] FillArrays v1.13.0
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [46192b85] GPUArraysCore v0.2.0
  [3e5b6fbb] HostCPUFeatures v0.1.17
  [615f187c] IfElse v0.1.1
  [3587e190] InverseFunctions v0.1.17
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.7.0
  [ae98c720] Jieko v0.2.1
  [ef3ab10e] KLU v0.6.0
  [ba0b0d4f] Krylov v0.9.10
  [10f19ff3] LayoutPointers v0.1.17
  [5078a376] LazyArrays v2.6.1
⌅ [7ed4a6bd] LinearSolve v2.39.0
  [bdcacae8] LoopVectorization v0.12.171
  [1914dd2f] MacroTools v0.5.15
  [d125e4d3] ManualMemory v0.1.8
  [2e0e35c7] Moshi v0.3.5
  [46d2c3a1] MuladdMacro v0.2.4
  [6fe1bfb0] OffsetArrays v1.16.0
  [bac558e1] OrderedCollections v1.8.0
  [bbf590c4] OrdinaryDiffEqCore v1.20.0
  [d96e819e] Parameters v0.12.3
  [f517fe37] Polyester v0.7.16
  [1d0040c9] PolyesterWeave v0.2.2
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.31.1
  [f2c3362d] RecursiveFactorization v0.2.23
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.1
  [7e49a35a] RuntimeGeneratedFunctions v0.5.13
  [94e857df] SIMDTypes v0.1.0
  [476501e8] SLEEFPirates v0.6.43
⌅ [0bca4576] SciMLBase v2.77.2
⌅ [c0aeaf25] SciMLOperators v0.3.12
  [53ae85a6] SciMLStructures v1.7.0
  [efcf1570] Setfield v1.1.2
  [ce78b400] SimpleUnPack v1.1.0
  [e56a9233] Sparspak v0.3.9
  [aedffcd0] Static v1.2.0
  [0d7ed370] StaticArrayInterface v1.8.0
  [90137ffa] StaticArrays v1.9.13
  [1e83bf80] StaticArraysCore v1.4.3
  [10745b16] Statistics v1.11.1
  [7792a7ef] StrideArraysCore v0.5.7
  [2efcf032] SymbolicIndexingInterface v0.3.38
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
  [8290d209] ThreadingUtilities v0.5.2
  [d5829a12] TriangularSolve v0.2.1
  [781d530d] TruncatedStacktraces v1.4.0
  [3a884ed6] UnPack v1.0.2
  [3d5dd08c] VectorizationBase v0.21.71
  [1d5cc7b8] IntelOpenMP_jll v2025.0.4+0
  [856f044c] MKL_jll v2025.0.1+1
  [1317d2d5] oneTBB_jll v2022.0.0+0
  [0dad84c5] ArgTools v1.1.2
  [56f22d72] Artifacts v1.11.0
  [2a0f44e3] Base64 v1.11.0
  [ade2ca70] Dates v1.11.0
  [8ba89e20] Distributed v1.11.0
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching v1.11.0
  [9fa8497b] Future v1.11.0
  [b77e0a4c] InteractiveUtils v1.11.0
  [4af54fe1] LazyArtifacts v1.11.0
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2 v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.11.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.11.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [6462fe0b] Sockets v1.11.0
  [2f01184e] SparseArrays v1.11.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test v1.11.0
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] LibCURL_jll v8.6.0+0
  [e37daf67] LibGit2_jll v1.7.2+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.6+0
  [14a3606d] MozillaCACerts_jll v2023.12.12
  [4536629a] OpenBLAS_jll v0.3.27+1
  [bea87d4a] SuiteSparse_jll v7.7.0+0
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.11.0+0
  [8e850ede] nghttp2_jll v1.59.0+0
  [3f19e933] p7zip_jll v17.4.0+2
LinearSolve.jl v3.7.0
(PositiveIntegrators) pkg> st -m
Project PositiveIntegrators v0.2.9-DEV
Status `~/.julia/dev/PositiveIntegrators/Manifest.toml`
  [47edcb42] ADTypes v1.14.0
  [7d9f7c33] Accessors v0.1.42
  [79e6a3ab] Adapt v4.3.0
  [4fba245c] ArrayInterface v7.18.0
  [4c555306] ArrayLayouts v1.11.1
  [62783981] BitTwiddlingConvenienceFunctions v0.1.6
  [2a0fbf3d] CPUSummary v0.2.6
  [d360d2e6] ChainRulesCore v1.25.1
  [fb6a15b2] CloseOpenIntervals v0.1.13
  [38540f10] CommonSolve v0.2.4
  [f70d9fcc] CommonWorldInvalidations v1.0.0
  [34da2185] Compat v4.16.0
  [a33af91c] CompositionsBase v0.1.2
  [2569d6c7] ConcreteStructs v0.2.3
  [187b0558] ConstructionBase v1.5.8
  [adafc99b] CpuId v0.3.1
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.22
  [e2d170a0] DataValueInterfaces v1.0.0
  [2b5f629d] DiffEqBase v6.167.0
  [ffbed154] DocStringExtensions v0.9.3
  [4e289a0a] EnumX v1.0.4
  [f151be2c] EnzymeCore v0.8.8
  [e2ba6199] ExprTools v0.1.10
  [55351af7] ExproniconLite v0.10.14
  [7034ab61] FastBroadcast v0.3.5
  [9aa1b823] FastClosures v0.3.2
  [a4df4552] FastPower v1.1.1
  [1a297f60] FillArrays v1.13.0
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [46192b85] GPUArraysCore v0.2.0
  [615f187c] IfElse v0.1.1
  [3587e190] InverseFunctions v0.1.17
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.7.0
  [ae98c720] Jieko v0.2.1
  [ba0b0d4f] Krylov v0.9.10
  [10f19ff3] LayoutPointers v0.1.17
  [5078a376] LazyArrays v2.6.1
  [7ed4a6bd] LinearSolve v3.7.0
  [1914dd2f] MacroTools v0.5.15
  [d125e4d3] ManualMemory v0.1.8
  [2e0e35c7] Moshi v0.3.5
  [46d2c3a1] MuladdMacro v0.2.4
  [bac558e1] OrderedCollections v1.8.0
  [bbf590c4] OrdinaryDiffEqCore v1.20.0
  [d96e819e] Parameters v0.12.3
  [f517fe37] Polyester v0.7.16
  [1d0040c9] PolyesterWeave v0.2.2
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.31.1
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.1
  [7e49a35a] RuntimeGeneratedFunctions v0.5.13
  [94e857df] SIMDTypes v0.1.0
  [0bca4576] SciMLBase v2.79.0
  [c0aeaf25] SciMLOperators v0.3.13
  [53ae85a6] SciMLStructures v1.7.0
  [efcf1570] Setfield v1.1.2
  [ce78b400] SimpleUnPack v1.1.0
  [aedffcd0] Static v1.2.0
  [0d7ed370] StaticArrayInterface v1.8.0
  [90137ffa] StaticArrays v1.9.13
  [1e83bf80] StaticArraysCore v1.4.3
  [10745b16] Statistics v1.11.1
  [7792a7ef] StrideArraysCore v0.5.7
  [2efcf032] SymbolicIndexingInterface v0.3.38
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
  [8290d209] ThreadingUtilities v0.5.2
  [781d530d] TruncatedStacktraces v1.4.0
  [3a884ed6] UnPack v1.0.2
  [1d5cc7b8] IntelOpenMP_jll v2025.0.4+0
  [856f044c] MKL_jll v2025.0.1+1
  [1317d2d5] oneTBB_jll v2022.0.0+0
  [0dad84c5] ArgTools v1.1.2
  [56f22d72] Artifacts v1.11.0
  [2a0f44e3] Base64 v1.11.0
  [ade2ca70] Dates v1.11.0
  [8ba89e20] Distributed v1.11.0
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching v1.11.0
  [9fa8497b] Future v1.11.0
  [b77e0a4c] InteractiveUtils v1.11.0
  [4af54fe1] LazyArtifacts v1.11.0
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2 v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.11.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.11.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [6462fe0b] Sockets v1.11.0
  [2f01184e] SparseArrays v1.11.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] LibCURL_jll v8.6.0+0
  [e37daf67] LibGit2_jll v1.7.2+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.6+0
  [14a3606d] MozillaCACerts_jll v2023.12.12
  [4536629a] OpenBLAS_jll v0.3.27+1
  [bea87d4a] SuiteSparse_jll v7.7.0+0
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.11.0+0
  [8e850ede] nghttp2_jll v1.59.0+0
  [3f19e933] p7zip_jll v17.4.0+2

However, I noticed that some libraries are not needed anymore for LinearSolve.jl v3. This is the diff of the versions according to the package manager when updating from LinearSolve.jl v2 to v3:

diff
(PositiveIntegrators) pkg> up
    Updating registry at `~/.julia/registries/General.toml`
    Updating `~/.julia/dev/PositiveIntegrators/Project.toml`
  [7ed4a6bd]  LinearSolve v2.39.0  v3.7.0
  [0bca4576]  SciMLBase v2.77.2  v2.79.0
    Updating `~/.julia/dev/PositiveIntegrators/Manifest.toml`
  [29a986be] - FastLapackInterface v2.0.4
  [3e5b6fbb] - HostCPUFeatures v0.1.17
  [ef3ab10e] - KLU v0.6.0
  [7ed4a6bd]  LinearSolve v2.39.0  v3.7.0
  [bdcacae8] - LoopVectorization v0.12.171
  [6fe1bfb0] - OffsetArrays v1.16.0
  [f2c3362d] - RecursiveFactorization v0.2.23
  [476501e8] - SLEEFPirates v0.6.43
  [0bca4576]  SciMLBase v2.77.2  v2.79.0
  [c0aeaf25]  SciMLOperators v0.3.12  v0.3.13
  [e56a9233] - Sparspak v0.3.9
  [d5829a12] - TriangularSolve v0.2.1
  [3d5dd08c] - VectorizationBase v0.21.71
  [8dfed614] - Test v1.11.0

I don't know if this could be somehow related to one of these packages?

@ranocha
Copy link
Member

ranocha commented Mar 24, 2025

Thanks. As far as I can tell, the other packages should not be involved - we use the UMFPACK solver also with LinearSolve.jl v2, don't we? Maybe we need to check the LinearSolve.jl code carefully to see whether there was any change except moving the sparse solver stuff to another file?

@JoshuaLampert
Copy link
Member

Thanks. As far as I can tell, the other packages should not be involved - we use the UMFPACK solver also with LinearSolve.jl v2, don't we?

Yes, I think so - or at least checking linsolve.cacheval gave SparseArrays.UMFPACK.UmfpackLU in both cases.

@ranocha
Copy link
Member

ranocha commented Mar 27, 2025

Will be fixed by SciML/LinearSolve.jl#591

@JoshuaLampert
Copy link
Member

Will be fixed by SciML/LinearSolve.jl#591

Oh wow, so the extension didn't really specialize the pattern_changed function for sparse matrices and some fallback was called? Thanks a lot for looking into this and finding the culprit!

@ranocha
Copy link
Member

ranocha commented Mar 27, 2025

Yeah, that was the issue.

@ranocha ranocha closed this in #153 Mar 28, 2025
@ranocha ranocha deleted the compathelper/new_version/2025-02-06-01-27-57-156-03022304862 branch May 16, 2025 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants