Move utilities to SciMLBase #1197

jClugstor · 2025-08-18T16:44:53Z

Checklist

Appropriate tests were added
Any code changes were done in a way that does not break public API
All documentation related to code changes were updated
The new code follows the
contributor guidelines, in particular the SciML Style Guide and
COLPRAC.
Any new documentation only uses public API

Additional context

This moves most of the utilities used by the solve function to SciMLBase. This will allow other packages to use some of the utilities useful for solve without depending on DiffEqBase. Specifically NonlinearSolveBase will need to use several of these functions in order to move away from depending on DiffEqBase.
(SciML/SciMLBase.jl#1110)

jClugstor · 2025-08-21T19:46:24Z

@ChrisRackauckas

ChrisRackauckas · 2025-08-21T21:41:24Z

Core tests fail

jClugstor · 2025-08-21T22:50:24Z

Yeah it's looking for functions that will be in SciMLBase but aren't there yet. I can confirm that on this branch and with SciMLBase master the Core tests pass, so I don't think SciMLBase needs anymore changes at least.

jClugstor · 2025-08-28T18:08:44Z

@ChrisRackauckas downstream 2 is failing because OrdinaryDiffEq.OrdinaryDiffEqCache no longer works, and MultiScaleArrays uses that. The DelayDiffEq interface tests are also failing, I'm not sure if that's due to this yet though.

bclyons12 · 2025-08-28T18:10:36Z

Without this, there are compatibility issues leading to precompilation errors from the automatic upgrade of SciMLBase. For example:

(@v1.11) pkg> activate --temp
  Activating new project at `/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_QyBdRH`

(jl_QyBdRH) pkg> add NonlinearSolve
   Resolving package versions...
   Installed DataStructures ─ v0.19.1
    Updating `/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_QyBdRH/Project.toml`
  [8913a72c] + NonlinearSolve v4.10.0
    Updating `/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_QyBdRH/Manifest.toml`
  [47edcb42] + ADTypes v1.17.0
  [7d9f7c33] + Accessors v0.1.42
  [79e6a3ab] + Adapt v4.3.0
  [4fba245c] + ArrayInterface v7.19.0
  [4c555306] + ArrayLayouts v1.11.2
  [62783981] + BitTwiddlingConvenienceFunctions v0.1.6
  [70df07ce] + BracketingNonlinearSolve v1.3.0
  [2a0fbf3d] + CPUSummary v0.2.7
  [d360d2e6] + ChainRulesCore v1.26.0
  [fb6a15b2] + CloseOpenIntervals v0.1.13
  [38540f10] + CommonSolve v0.2.4
  [bbf7d656] + CommonSubexpressions v0.3.1
  [f70d9fcc] + CommonWorldInvalidations v1.0.0
  [34da2185] + Compat v4.18.0
  [a33af91c] + CompositionsBase v0.1.2
  [2569d6c7] + ConcreteStructs v0.2.3
  [187b0558] + ConstructionBase v1.6.0
  [adafc99b] + CpuId v0.3.1
  [864edb3b] + DataStructures v0.19.1
  [2b5f629d] + DiffEqBase v6.183.2
  [163ba53b] + DiffResults v1.1.0
  [b552c78f] + DiffRules v1.15.1
  [a0c0ee7d] + DifferentiationInterface v0.7.7
  [ffbed154] + DocStringExtensions v0.9.5
  [4e289a0a] + EnumX v1.0.5
  [f151be2c] + EnzymeCore v0.8.12
  [e2ba6199] + ExprTools v0.1.10
  [55351af7] + ExproniconLite v0.10.14
  [7034ab61] + FastBroadcast v0.3.5
  [9aa1b823] + FastClosures v0.3.2
  [a4df4552] + FastPower v1.1.3
  [1a297f60] + FillArrays v1.13.0
  [6a86dc24] + FiniteDiff v2.28.1
  [f6369f11] + ForwardDiff v1.1.0
  [069b7b12] + FunctionWrappers v1.1.3
  [77dc65aa] + FunctionWrappersWrappers v0.1.3
  [46192b85] + GPUArraysCore v0.2.0
  [615f187c] + IfElse v0.1.1
  [3587e190] + InverseFunctions v0.1.17
  [92d709cd] + IrrationalConstants v0.2.4
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.7.1
  [ae98c720] + Jieko v0.2.1
  [ba0b0d4f] + Krylov v0.10.1
  [10f19ff3] + LayoutPointers v0.1.17
  [5078a376] + LazyArrays v2.6.2
  [87fe0de2] + LineSearch v0.1.4
  [7ed4a6bd] + LinearSolve v3.37.0
  [2ab3a3ac] + LogExpFunctions v0.3.29
  [1914dd2f] + MacroTools v0.5.16
  [d125e4d3] + ManualMemory v0.1.8
  [bb5d69b7] + MaybeInplace v0.1.4
  [2e0e35c7] + Moshi v0.3.7
  [46d2c3a1] + MuladdMacro v0.2.4
  [77ba4419] + NaNMath v1.1.3
  [8913a72c] + NonlinearSolve v4.10.0
  [be0214bd] + NonlinearSolveBase v1.14.0
  [5959db7a] + NonlinearSolveFirstOrder v1.7.0
  [9a2c21bd] + NonlinearSolveQuasiNewton v1.8.0
  [26075421] + NonlinearSolveSpectralMethods v1.3.0
  [bac558e1] + OrderedCollections v1.8.1
  [d96e819e] + Parameters v0.12.3
  [f517fe37] + Polyester v0.7.18
  [1d0040c9] + PolyesterWeave v0.2.2
  [d236fae5] + PreallocationTools v0.4.33
⌅ [aea7be01] + PrecompileTools v1.2.1
  [21216c6a] + Preferences v1.5.0
  [3cdcf5f2] + RecipesBase v1.3.4
  [731186ca] + RecursiveArrayTools v3.37.1
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.1
  [7e49a35a] + RuntimeGeneratedFunctions v0.5.15
  [94e857df] + SIMDTypes v0.1.0
  [0bca4576] + SciMLBase v2.114.0
  [19f34311] + SciMLJacobianOperators v0.1.8
  [c0aeaf25] + SciMLOperators v1.6.0
  [53ae85a6] + SciMLStructures v1.7.0
  [efcf1570] + Setfield v1.1.2
  [727e6d20] + SimpleNonlinearSolve v2.7.0
  [0a514795] + SparseMatrixColorings v0.4.21
  [276daf66] + SpecialFunctions v2.5.1
  [aedffcd0] + Static v1.2.0
  [0d7ed370] + StaticArrayInterface v1.8.0
  [90137ffa] + StaticArrays v1.9.14
  [1e83bf80] + StaticArraysCore v1.4.3
  [10745b16] + Statistics v1.11.1
  [7792a7ef] + StrideArraysCore v0.5.8
  [2efcf032] + SymbolicIndexingInterface v0.3.43
  [8290d209] + ThreadingUtilities v0.5.5
  [a759f4b9] + TimerOutputs v0.5.29
  [781d530d] + TruncatedStacktraces v1.4.0
  [3a884ed6] + UnPack v1.0.2
  [1d5cc7b8] + IntelOpenMP_jll v2025.2.0+0
  [856f044c] + MKL_jll v2025.2.0+0
  [efe28fd5] + OpenSpecFun_jll v0.5.6+0
  [1317d2d5] + oneTBB_jll v2022.0.0+0
  [0dad84c5] + ArgTools v1.1.2
  [56f22d72] + Artifacts v1.11.0
  [2a0f44e3] + Base64 v1.11.0
  [ade2ca70] + Dates v1.11.0
  [8ba89e20] + Distributed v1.11.0
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching v1.11.0
  [9fa8497b] + Future v1.11.0
  [b77e0a4c] + InteractiveUtils v1.11.0
  [4af54fe1] + LazyArtifacts v1.11.0
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2 v1.11.0
  [8f399da3] + Libdl v1.11.0
  [37e2e46d] + LinearAlgebra v1.11.0
  [56ddb016] + Logging v1.11.0
  [d6f4376e] + Markdown v1.11.0
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.11.0
  [de0858da] + Printf v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization v1.11.0
  [6462fe0b] + Sockets v1.11.0
  [2f01184e] + SparseArrays v1.11.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] + LibCURL_jll v8.6.0+0
  [e37daf67] + LibGit2_jll v1.7.2+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.6+0
  [14a3606d] + MozillaCACerts_jll v2023.12.12
  [4536629a] + OpenBLAS_jll v0.3.27+1
  [05823500] + OpenLibm_jll v0.8.5+0
  [bea87d4a] + SuiteSparse_jll v7.7.0+0
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850b90] + libblastrampoline_jll v5.11.0+0
  [8e850ede] + nghttp2_jll v1.59.0+0
  [3f19e933] + p7zip_jll v17.4.0+2
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling project...
  ? DiffEqBase
  ? DiffEqBase → DiffEqBaseChainRulesCoreExt
  ? DiffEqBase → DiffEqBaseForwardDiffExt
  ? DiffEqBase → DiffEqBaseSparseArraysExt
  ? NonlinearSolveBase → NonlinearSolveBaseDiffEqBaseExt
  ? SimpleNonlinearSolve → SimpleNonlinearSolveDiffEqBaseExt
  ? NonlinearSolveSpectralMethods
  ? NonlinearSolveQuasiNewton
  ? NonlinearSolveFirstOrder
  ? NonlinearSolveSpectralMethods → NonlinearSolveSpectralMethodsForwardDiffExt
  ? NonlinearSolveQuasiNewton → NonlinearSolveQuasiNewtonForwardDiffExt
  ? NonlinearSolve
  8 dependencies successfully precompiled in 19 seconds. 175 already precompiled.
  12 dependencies failed but may be precompilable after restarting julia
  12 dependencies had output during precompilation:
┌ DiffEqBase → DiffEqBaseForwardDiffExt
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ NonlinearSolve
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ SimpleNonlinearSolve → SimpleNonlinearSolveDiffEqBaseExt
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ DiffEqBase
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ NonlinearSolveBase → NonlinearSolveBaseDiffEqBaseExt
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ DiffEqBase → DiffEqBaseChainRulesCoreExt
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ NonlinearSolveFirstOrder
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ NonlinearSolveSpectralMethods → NonlinearSolveSpectralMethodsForwardDiffExt
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ NonlinearSolveQuasiNewton → NonlinearSolveQuasiNewtonForwardDiffExt
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ DiffEqBase → DiffEqBaseSparseArraysExt
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ NonlinearSolveSpectralMethods
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└  
┌ NonlinearSolveQuasiNewton
│  WARNING: Method definition solve(SciMLBase.EnsembleProblem{T, T2, T3, T4, T5} where T5 where T4 where T3 where T2 where T, Any...) in module SciMLBase at /Users/lyons/.julia/packages/SciMLBase/wQHrI/src/ensemble/basic_ensemble_solve.jl:356 overwritten in module DiffEqBase at /Users/lyons/.julia/packages/DiffEqBase/AfBQT/src/solve.jl:1224.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.

jClugstor · 2025-08-28T18:47:15Z

It's because solve for EnsembleProblems is duplicated and becomes overwritten. Maybe better to fix the precompilation issue first then handle this since it's such a huge PR. #1201

jClugstor · 2025-08-29T19:23:18Z

@ChrisRackauckas as far as I can tell none of the test failures are caused by this, so I think this is ready to look at.

jClugstor force-pushed the move_to_SciMLBase branch from f7c3a4f to bd458cd Compare August 21, 2025 22:35

jClugstor added 14 commits August 29, 2025 08:56

excise stuff in solve and utils

6b7cff1

import needed things from SciMLBase

f8f5d97

get rid of Distributions extension

ce72abc

excise extension things that are now in SciMLBase

ed05d57

import errors from SciMLBase

099f3d6

import NonConcreteEltypeError

bbba8e2

get rid of comments

d666929

remove _concrete_solve_adjoint and _concrete_solve_forward

ba4b134

add imports for _concrete solves

9d9d672

remove redundant mooncake rules

dd5141b

format solve

ee431a9

change errors to reference SciMLBase

8b82b83

fix typo

2be3a5b

fix unitful tests

a843902

jClugstor force-pushed the move_to_SciMLBase branch from 2ed3616 to a843902 Compare August 29, 2025 12:57

remove _vec, already in SciMLBase

f845a85

jClugstor force-pushed the move_to_SciMLBase branch from d405d1d to f845a85 Compare August 29, 2025 14:01

ChrisRackauckas merged commit f9157be into SciML:master Aug 29, 2025
38 of 48 checks passed

Krastanov mentioned this pull request Sep 1, 2025

fix for breaking internal change to DiffEqBase.promote_dual qojulia/QuantumOptics.jl#461

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Uh oh!

Move utilities to SciMLBase #1197

Move utilities to SciMLBase #1197

Uh oh!

jClugstor commented Aug 18, 2025

Uh oh!

jClugstor commented Aug 21, 2025

Uh oh!

ChrisRackauckas commented Aug 21, 2025

Uh oh!

jClugstor commented Aug 21, 2025

Uh oh!

jClugstor commented Aug 28, 2025

Uh oh!

bclyons12 commented Aug 28, 2025

Uh oh!

jClugstor commented Aug 28, 2025

Uh oh!

jClugstor commented Aug 29, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Move utilities to SciMLBase #1197

Move utilities to SciMLBase #1197

Uh oh!

Conversation

jClugstor commented Aug 18, 2025

Checklist

Additional context

Uh oh!

jClugstor commented Aug 21, 2025

Uh oh!

ChrisRackauckas commented Aug 21, 2025

Uh oh!

jClugstor commented Aug 21, 2025

Uh oh!

jClugstor commented Aug 28, 2025

Uh oh!

bclyons12 commented Aug 28, 2025

Uh oh!

jClugstor commented Aug 28, 2025

Uh oh!

jClugstor commented Aug 29, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants