Misdetection of Automatic Differentiation of the Solve Process

[GrantHecht]

Describe the bug 🐞

I frequently need to solve nonlinear systems that depend on both the values and first derivatives of Chebyshev polynomial interpolations (among other quantities). The analytical first derivatives are provided through FastChebInterp.jl. In general, most elements of my Jacobian can be computed analytically, but some second derivatives of the Chebyshev interpolation must be obtained using automatic differentiation (AD). To facilitate this, I typically pass the problem parameters to NonlinearSolve as a Tuple, where one element is a struct containing both the Chebyshev interpolation and any AD preparation needed to compute the second derivatives inside the Jacobian (with all other Jacobian elements computed analytically).

This setup has worked reliably in the past. However, after a recent update, it appears that NonlinearSolve.jl is now incorrectly detecting differentiation of the solve with respect to the problem parameters. Specifically, both the function and Jacobian are being evaluated using ForwardDiff.Dual types instead of Float64, even though no differentiation of the solver itself is intended.

Expected behavior

The solve process should be performed with element types consistent with the element type of the initial guess.

Minimal Reproducible Example 👇

Below is a simple example that resembles my setup and produces this issue. Observe from the output/stacktrace, the type of u when the Jacobian is evaluated is Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(complex_function), Float64}, Float64, 1}}, prior to the error getting thrown due to inconsistent types between the AD prep and execution.

using NonlinearSolve

import DifferentiationInterface as AD
import ForwardDiff

complex_function(x) = sin(x)

function nl_eqs!(F,u,p)
    a = p[1]
    F[1] = complex_function(u[1])
    F[2] = a*u[1] + u[2]
    return nothing
end

function nl_jac!(J,u,p)
    a, s = p
    println(typeof(u))
    J[1,1] = AD.derivative(complex_function, s.prep, s.backend, u[1])
    J[1,2] = 0.0
    J[2,1] = a
    J[2,2] = 1.0
    return nothing
end

struct StuffForComplexFunction{P,B}
    prep::P
    backend::B
    function StuffForComplexFunction(prep::P,backend::B) where{P,B}
        new{P,B}(prep,backend)
    end
end

function main()
    # Prep for AD
    backend = AD.AutoForwardDiff()
    prep = AD.prepare_derivative(complex_function, backend, 0.1)

    ps = (1.0, StuffForComplexFunction(prep,backend))

    prob = NonlinearProblem(
        NonlinearFunction{true,SciMLBase.FullSpecialize}(nl_eqs!; jac=nl_jac!),
        [0.1,0.1], ps;
    )

    sol = solve(prob, TrustRegion(;autodiff=nothing))
end

main()

Error & Stacktrace ⚠️

Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(complex_function), Float64}, Float64, 1}}
ERROR: LoadError: PreparationMismatchError (inconsistent types between preparation and execution):
  - f: ✅
  - backend: ✅
  - x: ❌
    - prep: Float64
    - exec: ForwardDiff.Dual{ForwardDiff.Tag{typeof(complex_function), Float64}, Float64, 1}
  - contexts: ✅
If you are confident that this check is superfluous, you can disable it by running preparation with the keyword argument `strict=Val(false)` inside DifferentiationInterface.

Stacktrace:
  [1] check_prep(::Function, ::DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgDerivativePrep{…}, ::AutoForwardDiff{…}, ::ForwardDiff.Dual{…})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/L0TGS/src/utils/prep.jl:169
  [2] derivative(::typeof(complex_function), ::DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgDerivativePrep{…}, ::AutoForwardDiff{…}, ::ForwardDiff.Dual{…})
    @ DifferentiationInterfaceForwardDiffExt ~/.julia/packages/DifferentiationInterface/L0TGS/ext/DifferentiationInterfaceForwardDiffExt/onearg.jl:264
  [3] nl_jac!(J::Matrix{ForwardDiff.Dual{…}}, u::Vector{ForwardDiff.Dual{…}}, p::Tuple{Float64, StuffForComplexFunction{…}})
    @ Main ~/Documents/Projects/Code/NonlinearSolveADVarsInParamsError/error_mwe.jl:19
  [4] (::NonlinearSolveBase.JacobianCache{…})(u::Vector{…})
    @ NonlinearSolveBase ~/.julia/packages/NonlinearSolveBase/2E600/src/jacobian.jl:159
  [5] step!(cache::NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithmCache{…}; recompute_jacobian::Nothing)
    @ NonlinearSolveFirstOrder ~/.julia/packages/NonlinearSolveFirstOrder/gwWsK/src/solve.jl:238
  [6] step!
    @ ~/.julia/packages/NonlinearSolveFirstOrder/gwWsK/src/solve.jl:232 [inlined]
  [7] #step!#173
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:500 [inlined]
  [8] step!
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:494 [inlined]
  [9] solve!
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:240 [inlined]
 [10] __solve(::NonlinearProblem{…}, ::GeneralizedFirstOrderAlgorithm{…}; kwargs::@Kwargs{…})
    @ NonlinearSolveBase ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:228
 [11] __solve
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:223 [inlined]
 [12] #solve_call#143
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:154 [inlined]
 [13] solve_call
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:120 [inlined]
 [14] #solve_up#142
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:115 [inlined]
 [15] solve_up
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:101 [inlined]
 [16] #solve#141
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:81 [inlined]
 [17] solve
    @ ~/.julia/packages/NonlinearSolveBase/2E600/src/solve.jl:47 [inlined]
 [18] main()
    @ Main ~/Documents/Projects/Code/NonlinearSolveADVarsInParamsError/error_mwe.jl:46
 [19] top-level scope
    @ ~/Documents/Projects/Code/NonlinearSolveADVarsInParamsError/error_mwe.jl:49
 [20] include(mapexpr::Function, mod::Module, _path::String)
    @ Base ./Base.jl:307
 [21] top-level scope
    @ REPL[9]:1
in expression starting at /Users/granthec/Documents/Projects/Code/NonlinearSolveADVarsInParamsError/error_mwe.jl:49
Some type information was truncated. Use `show(err)` to see complete types.

Environment (please complete the following information):

• Output of using Pkg; Pkg.status()

Status `~/Documents/Projects/Code/NonlinearSolveADVarsInParamsError/Project.toml`
  [a0c0ee7d] DifferentiationInterface v0.7.9
  [f6369f11] ForwardDiff v1.2.2
  [8913a72c] NonlinearSolve v4.12.0

• Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)

Status `~/Documents/Projects/Code/NonlinearSolveADVarsInParamsError/Manifest.toml`
  [47edcb42] ADTypes v1.18.0
  [7d9f7c33] Accessors v0.1.42
  [79e6a3ab] Adapt v4.4.0
  [4fba245c] ArrayInterface v7.22.0
  [4c555306] ArrayLayouts v1.12.0
  [70df07ce] BracketingNonlinearSolve v1.5.0
  [d360d2e6] ChainRulesCore v1.26.0
  [38540f10] CommonSolve v0.2.4
  [bbf7d656] CommonSubexpressions v0.3.1
  [34da2185] Compat v4.18.1
  [a33af91c] CompositionsBase v0.1.2
  [2569d6c7] ConcreteStructs v0.2.3
  [187b0558] ConstructionBase v1.6.0
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [a0c0ee7d] DifferentiationInterface v0.7.9
  [ffbed154] DocStringExtensions v0.9.5
  [4e289a0a] EnumX v1.0.5
  [f151be2c] EnzymeCore v0.8.14
  [e2ba6199] ExprTools v0.1.10
  [55351af7] ExproniconLite v0.10.14
  [9aa1b823] FastClosures v0.3.2
  [1a297f60] FillArrays v1.14.0
  [6a86dc24] FiniteDiff v2.29.0
  [f6369f11] ForwardDiff v1.2.2
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [46192b85] GPUArraysCore v0.2.0
  [3587e190] InverseFunctions v0.1.17
  [92d709cd] IrrationalConstants v0.2.6
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.7.1
  [ae98c720] Jieko v0.2.1
  [ba0b0d4f] Krylov v0.10.2
  [5078a376] LazyArrays v2.8.0
  [87fe0de2] LineSearch v0.1.4
  [7ed4a6bd] LinearSolve v3.45.0
  [2ab3a3ac] LogExpFunctions v0.3.29
  [1914dd2f] MacroTools v0.5.16
  [bb5d69b7] MaybeInplace v0.1.4
  [2e0e35c7] Moshi v0.3.7
  [77ba4419] NaNMath v1.1.3
  [8913a72c] NonlinearSolve v4.12.0
  [be0214bd] NonlinearSolveBase v2.0.0
  [5959db7a] NonlinearSolveFirstOrder v1.9.0
  [9a2c21bd] NonlinearSolveQuasiNewton v1.10.0
  [26075421] NonlinearSolveSpectralMethods v1.5.0
  [d236fae5] PreallocationTools v0.4.34
  [aea7be01] PrecompileTools v1.3.3
  [21216c6a] Preferences v1.5.0
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.39.0
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.1
  [7e49a35a] RuntimeGeneratedFunctions v0.5.15
  [0bca4576] SciMLBase v2.122.0
  [19f34311] SciMLJacobianOperators v0.1.11
  [c0aeaf25] SciMLOperators v1.9.0
  [431bcebd] SciMLPublic v1.0.0
  [53ae85a6] SciMLStructures v1.7.0
  [efcf1570] Setfield v1.1.2
  [727e6d20] SimpleNonlinearSolve v2.9.0
  [276daf66] SpecialFunctions v2.6.1
  [90137ffa] StaticArrays v1.9.15
  [1e83bf80] StaticArraysCore v1.4.4
  [10745b16] Statistics v1.11.1
  [2efcf032] SymbolicIndexingInterface v0.3.46
  [a759f4b9] TimerOutputs v0.5.29
  [3a884ed6] UnPack v1.0.2
  [1d5cc7b8] IntelOpenMP_jll v2025.2.0+0
  [856f044c] MKL_jll v2025.2.0+0
  [efe28fd5] OpenSpecFun_jll v0.5.6+0
  [1317d2d5] oneTBB_jll v2022.0.0+1
  [0dad84c5] ArgTools v1.1.2
  [56f22d72] Artifacts v1.11.0
  [2a0f44e3] Base64 v1.11.0
  [ade2ca70] Dates v1.11.0
  [8ba89e20] Distributed v1.11.0
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching v1.11.0
  [9fa8497b] Future v1.11.0
  [b77e0a4c] InteractiveUtils v1.11.0
  [ac6e5ff7] JuliaSyntaxHighlighting v1.12.0
  [4af54fe1] LazyArtifacts v1.11.0
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2 v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.12.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [ca575930] NetworkOptions v1.3.0
  [44cfe95a] Pkg v1.12.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [6462fe0b] Sockets v1.11.0
  [2f01184e] SparseArrays v1.12.0
  [f489334b] StyledStrings v1.11.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.3.0+1
  [deac9b47] LibCURL_jll v8.11.1+1
  [e37daf67] LibGit2_jll v1.9.0+0
  [29816b5a] LibSSH2_jll v1.11.3+1
  [14a3606d] MozillaCACerts_jll v2025.5.20
  [4536629a] OpenBLAS_jll v0.3.29+0
  [05823500] OpenLibm_jll v0.8.7+0
  [458c3c95] OpenSSL_jll v3.5.1+0
  [bea87d4a] SuiteSparse_jll v7.8.3+2
  [83775a58] Zlib_jll v1.3.1+2
  [8e850b90] libblastrampoline_jll v5.15.0+0
  [8e850ede] nghttp2_jll v1.64.0+1
  [3f19e933] p7zip_jll v17.5.0+2

• Output of versioninfo()

Julia Version 1.12.1
Commit ba1e628ee49 (2025-10-17 13:02 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 10 × Apple M1 Pro
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m1)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 8 virtual cores)
Environment:
  JULIA_PARDISO = `/Users/granthec/PARDISO/panua-pardiso-20240630-mac_arm64/lib`

[ChrisRackauckas]

This should get a specialization on SciMLBase.anytypedual. Does StuffForComplexFunction have an abstract type on it that we should be consider opting out on?

[GrantHecht]

In my use case, the struct that is represented by StuffForComplexFunction does not have an abstract type (it's just a very problem-specific struct defined to pack up things used to evaluate the function/jac efficiently). Since this seems to be triggered by the type of prep (DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPusforwardPrep{...}), would you be able to opt out of anything that's a subtype of DifferentiationInterface.DerivativePrep or DifferentiationInterface.Prep?

[ChrisRackauckas]

Oh definitely. SciMLBase.anytypedual(::DifferentiationInterface.Prep) = Any is something we should add. We directly did this for ForwardDiff caches ("config"s) already https://github.com/SciML/SciMLBase.jl/blob/master/ext/SciMLBaseForwardDiffExt.jl#L353-L361 so we should add the same type of dispatches on Prep.