Fit setting with ADAM

[rzhli]

Since Flux has changed it's interface of learning rate scheduler
opt = Flux.Optimiser(ADAM(0.05), ExpDecay(1.0, 0.985, 20)) in
https://github.com/viniviena/ude_chromatography/blob/f12e20d349957e579074bad35de5edee9c187fd6/UDE_paper_chromatography/PDE_gradients_lux_ude.jl#L314C1-L314C59
failed, I have tried with

using OptimizationOptimisers
opt = Adam(0.05, (1.0, 0.985)

which errors as:

ERROR: Function argument passed to autodiff cannot be proven readonly.
If the the function argument cannot contain derivative data, instead call autodiff(Mode, Const(f), ...)
See https://enzyme.mit.edu/index.fcgi/julia/stable/faq/#Activity-of-temporary-storage for more information.

[viniviena]

Hey. Thanks for bringing it up. That seems to be dispatching autodiff things with enzyme. Which versions of the deps are you using now?

[rzhli]

(chromatography) pkg> st
Status `~/文件/julia/chromatography/Project.toml`
  [0a1fb500] BlockDiagonals v0.1.42
  [b0b7db55] ComponentArrays v0.15.25
  [a93c6f00] DataFrames v1.7.0
⌃ [82cc6244] DataInterpolations v6.6.0
  [aae7a2af] DiffEqFlux v4.2.0
  [0c46a032] DifferentialEquations v7.16.0
  [31c24e10] Distributions v0.25.117
  [5b8099bc] DomainSets v0.7.15
  [38e38edf] GLM v1.9.0
  [2fda8390] LsqFit v0.15.0
  [b2108857] Lux v1.8.0
  [94925ecb] MethodOfLines v0.11.7
  [961ee093] ModelingToolkit v9.64.3
  [8913a72c] NonlinearSolve v4.4.0
  [7f7a1694] Optimization v4.1.1
  [36348300] OptimizationOptimJL v0.4.1
  [42dfb2eb] OptimizationOptimisers v0.3.7
  [1dea7af3] OrdinaryDiffEq v6.91.0
  [f0f68f2c] PlotlyJS v0.18.15
  [91a5bcdd] Plots v1.40.9
  [37e2e3b7] ReverseDiff v1.15.3
  [1ed8b502] SciMLSensitivity v7.74.0
  [f3b207a7] StatsPlots v0.15.7
  [6aa5eb33] TaylorSeries v0.18.3
  [3a884ed6] UnPack v1.0.2
  [fdbf4ff8] XLSX v0.10.4
⌃ [e88e6eb3] Zygote v0.6.75
Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> versioninfo()
Julia Version 1.11.3
Commit d63adeda50d (2025-01-21 19:42 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 12 × AMD Ryzen 5 5600H with Radeon Graphics
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 12 default, 0 interactive, 6 GC (on 12 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 12
  JULIA_PKG_SERVER = https://pkg.julialang.org

The full stack trace has been uploaded in discourse, you may have already noticed

[viniviena]

That's super weird. I dont know even why Enzyme is being called since AutoReverseDiff is being used for AD stuff. I am trying to make it work again with the same versions you have. I let you know as soon as I make any progress.

[rzhli]

why Enzyme is being called

Seems like they are moving towards to Enzyme.jl as descriped

Slight Detour: We have had several questions regarding if we will be considering any other AD system for the reverse-diff backend. For now we will stick to Zygote.jl, however once we have tested Lux extensively with Enzyme.jl, we will make the switch.

here

[viniviena]

It seems like we need to find the release corresponding to the switch and downgrade. Either that or finding a way of making it work with Enzyme.

[viniviena]

Hi!

Have you got it work with the issue? i saw that Chris tried to figure what the problem was.

[rzhli]

No, I also tried

#-----------------------Training Neural Network----------------------

# Dimensionless

t_train = c_exp_data[:, 1] ./ maximum(c_exp_data[:, 1])
y_train = c_exp_data[:, 2] ./ c_in
plot(t_train, y_train)

using SciMLSensitivity

function predict(p, t)
    sensealg = QuadratureAdjoint(; autojacvec = ReverseDiffVJP(true))
    # sensealg = InterpolatingAdjoint(; autojacvec = ReverseDiffVJP(true))
    prob = remake(chrom_prob_nn; tspan = extrema(t), p = p)

    sol = Array(
        solve(
            prob, FBDF(autodiff = false), abstol = 5e-7, reltol = 5e-7, 
            sensealg = sensealg, saveat = t
        )
    )
    return @view sol[Int(N_v / 2), :]
end

p = copy(params)
@time predict(p, t_train[1:5])

using Optimization, OptimizationOptimisers, Zygote

function train_one_round(p, t_train, y_train, opt, maxiters) 
    pred(p) = predict(p, t_train)
    loss(p) = MSELoss()(pred(p), y_train)

    adtype = Optimization.AutoZygote()
    optf = OptimizationFunction((p, _) -> loss(p), adtype)
    optprob = OptimizationProblem(optf, p)
    res = solve(optprob, opt; maxiters = maxiters)
    res.minimizer, state
end

MSELoss()(predict(p, t_train), y_train)

@time train_one_round(p, t_train[1:4], y_train[1:4], Adam(0.05, (1.0, 0.985)), 180)

still errors:

┌ Warning: Lux.apply(m::AbstractLuxLayer, x::AbstractArray{<:ReverseDiff.TrackedReal}, ps, st) input was corrected to Lux.apply(m::AbstractLuxLayer, x::ReverseDiff.TrackedArray}, ps, st).
│ 
│ 1. If this was not the desired behavior overload the dispatch on `m`.
│ 
│ 2. This might have performance implications. Check which layer was causing this problem using `Lux.Experimental.@debug_mode`.
└ @ LuxCoreArrayInterfaceReverseDiffExt ~/.julia/packages/LuxCore/Av7WJ/ext/LuxCoreArrayInterfaceReverseDiffExt.jl:9
┌ Warning: At t=0.0, dt was forced below floating point epsilon 5.0e-324, and step error estimate = 1.0. Aborting. There is either an error in your model specification or the true solution is unstable (or the true solution can not be represented in the precision of Float64).
└ @ SciMLBase ~/.julia/packages/SciMLBase/cwnDi/src/integrator_interface.jl:623

ERROR: Function argument passed to autodiff cannot be proven readonly.
If the the function argument cannot contain derivative data, instead call autodiff(Mode, Const(f), ...)
See https://enzyme.mit.edu/index.fcgi/julia/stable/faq/#Activity-of-temporary-storage for more information.
The potentially writing call is   store {} addrspace(10)* %.fca.0.0.0.0.1.extract, {} addrspace(10)** %.fca.0.0.0.0.1.gep, align 8, !dbg !66, !noalias !86, using   %.fca.0.0.0.0.1.gep = getelementptr inbounds { { { { { {} addrspace(10)*, {} addrspace(10)*, { [12 x [1 x [12 x double]]], [2 x {} addrspace(10)*] }, { { [1 x { {} addrspace(10)*, { i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* } }], [3 x {} addrspace(10)*], {} addrspace(10)*, {} addrspace(10)* }, {} addrspace(10)*, {} addrspace(10)* }, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, i8 }, { double, double }, i8, {} addrspace(10)*, { {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, { { [2 x i64], i64, [2 x i64], [2 x i64], i8, i8, [2 x i64] }, {} addrspace(10)*, {} addrspace(10)*, { i8 } }, { { {} addrspace(10)*, { i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* } }, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, i8, {} addrspace(10)*, { i64, {} addrspace(10)*, {} addrspace(10)* }, i8 }, i8, i64, {} addrspace(10)*, i32 }, { {} addrspace(10)*, { i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* } } }, [1 x { i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }] }, {} addrspace(10)*, i8, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, double, {} addrspace(10)* }, [1 x i8] }, { { { { { {} addrspace(10)*, {} addrspace(10)*, { [12 x [1 x [12 x double]]], [2 x {} addrspace(10)*] }, { { [1 x { {} addrspace(10)*, { i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* } }], [3 x {} addrspace(10)*], {} addrspace(10)*, {} addrspace(10)* }, {} addrspace(10)*, {} addrspace(10)* }, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, i8 }, { double, double }, i8, {} addrspace(10)*, { {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, { { [2 x i64], i64, [2 x i64], [2 x i64], i8, i8, [2 x i64] }, {} addrspace(10)*, {} addrspace(10)*, { i8 } }, { { {} addrspace(10)*, { i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* } }, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, i8, {} addrspace(10)*, { i64, {} addrspace(10)*, {} addrspace(10)* }, i8 }, i8, i64, {} addrspace(10)*, i32 }, { {} addrspace(10)*, { i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* } } }, [1 x { i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }] }, {} addrspace(10)*, i8, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, double, {} addrspace(10)* }, [1 x i8] }* %.innerparm, i64 0, i32 0, i32 0, i32 0, i32 0, i32 1, !dbg !66

Stacktrace:
  [1] NonlinearFunction
    @ ~/.julia/packages/SciMLBase/cwnDi/src/scimlfunctions.jl:0 [inlined]
  [2] diffejulia_NonlinearFunction_91795_inner_27wrap
    @ ~/.julia/packages/SciMLBase/cwnDi/src/scimlfunctions.jl:0
  [3] macro expansion
    @ ~/.julia/packages/Enzyme/txp7d/src/compiler.jl:5448 [inlined]
  [4] enzyme_call
    @ ~/.julia/packages/Enzyme/txp7d/src/compiler.jl:4986 [inlined]
  [5] CombinedAdjointThunk
    @ ~/.julia/packages/Enzyme/txp7d/src/compiler.jl:4858 [inlined]
  [6] autodiff
    @ ~/.julia/packages/Enzyme/txp7d/src/Enzyme.jl:503 [inlined]
  [7] autodiff
    @ ~/.julia/packages/Enzyme/txp7d/src/Enzyme.jl:524 [inlined]
  [8] value_and_pullback!(f!::NonlinearFunction{…}, y::Vector{…}, tx::Tuple{…}, prep::DifferentiationInterfaceEnzymeExt.EnzymeReverseTwoArgPullbackPrep{…}, backend::AutoEnzyme{…}, x::Vector{…}, ty::Tuple{…}, contexts::DifferentiationInterface.Constant{…})
    @ DifferentiationInterfaceEnzymeExt ~/.julia/packages/DifferentiationInterface/Yk2Kt/ext/DifferentiationInterfaceEnzymeExt/reverse_twoarg.jl:132
  [9] pullback!
    @ ~/.julia/packages/DifferentiationInterface/Yk2Kt/src/first_order/pullback.jl:441 [inlined]
 [10] #9
    @ ~/.julia/packages/SciMLJacobianOperators/W8DtG/src/SciMLJacobianOperators.jl:312 [inlined]
 [11] (::SciMLJacobianOperators.JacobianOperator{…})(v::Vector{…}, u::Vector{…}, p::ComponentVector{…})
    @ SciMLJacobianOperators ~/.julia/packages/SciMLJacobianOperators/W8DtG/src/SciMLJacobianOperators.jl:140
 [12] (::LineSearch.var"#4#6")(du::Vector{…}, u::Vector{…}, fu::Vector{…}, p::ComponentVector{…})
    @ LineSearch ~/.julia/packages/LineSearch/Ky1ZB/src/utils.jl:44
 [13] #10
    @ ~/.julia/packages/LineSearch/Ky1ZB/src/backtracking.jl:78 [inlined]
 [14] #14
    @ ~/.julia/packages/LineSearch/Ky1ZB/src/backtracking.jl:94 [inlined]
 [15] solve!(cache::LineSearch.BackTrackingCache{…}, u::Vector{…}, du::Vector{…})
    @ LineSearch ~/.julia/packages/LineSearch/Ky1ZB/src/backtracking.jl:97
 [16] step!(cache::NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithmCache{…}; recompute_jacobian::Nothing)
    @ NonlinearSolveFirstOrder ~/.julia/packages/NonlinearSolveFirstOrder/yeYrV/src/solve.jl:277
 [17] step!
    @ ~/.julia/packages/NonlinearSolveFirstOrder/yeYrV/src/solve.jl:224 [inlined]
 [18] #step!#163
    @ ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:253 [inlined]
 [19] step!
    @ ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:247 [inlined]
 [20] solve!(cache::NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithmCache{…})
    @ NonlinearSolveBase ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:18
 [21] __solve(::NonlinearProblem{…}, ::NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithm{…}; kwargs::@Kwargs{…})
    @ NonlinearSolveBase ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:6
 [22] macro expansion
    @ ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:173 [inlined]
 [23] __generated_polysolve(::NonlinearProblem{…}, ::NonlinearSolveBase.NonlinearSolvePolyAlgorithm{…}; stats::SciMLBase.NLStats, alias_u0::Bool, verbose::Bool, initializealg::NonlinearSolveBase.NonlinearSolveDefaultInit, kwargs::@Kwargs{…})
    @ NonlinearSolveBase ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:130
 [24] __generated_polysolve
    @ ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:130 [inlined]
 [25] #__solve#154
    @ ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:127 [inlined]
 [26] __solve
    @ ~/.julia/packages/NonlinearSolveBase/NTLPL/src/solve.jl:124 [inlined]
 [27] #solve_call#35
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:635 [inlined]
 [28] solve_call
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:592 [inlined]
 [29] #solve_up#44
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:1151 [inlined]
 [30] solve_up
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:1145 [inlined]
 [31] solve(prob::NonlinearProblem{…}, args::NonlinearSolveBase.NonlinearSolvePolyAlgorithm{…}; sensealg::Nothing, u0::Nothing, p::Nothing, wrap::Val{…}, kwargs::@Kwargs{…})
    @ DiffEqBase ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:1139
 [32] _initialize_dae!(integrator::OrdinaryDiffEqCore.ODEIntegrator{…}, prob::ODEProblem{…}, alg::BrownFullBasicInit{…}, isinplace::Val{…})
    @ OrdinaryDiffEqNonlinearSolve ~/.julia/packages/OrdinaryDiffEqNonlinearSolve/e4hoO/src/initialize_dae.jl:434
 [33] _initialize_dae!
    @ ~/.julia/packages/OrdinaryDiffEqCore/yWlnm/src/initialize_dae.jl:56 [inlined]
 [34] initialize_dae!
    @ ~/.julia/packages/OrdinaryDiffEqCore/yWlnm/src/initialize_dae.jl:40 [inlined]
 [35] initialize_dae!(integrator::OrdinaryDiffEqCore.ODEIntegrator{…})
    @ OrdinaryDiffEqCore ~/.julia/packages/OrdinaryDiffEqCore/yWlnm/src/initialize_dae.jl:40
 [36] __init(prob::ODEProblem{…}, alg::FBDF{…}, timeseries_init::Tuple{}, ts_init::Tuple{}, ks_init::Tuple{}, recompile::Type{…}; saveat::Tuple{}, tstops::Tuple{}, d_discontinuities::Tuple{}, save_idxs::Nothing, save_everystep::Bool, save_on::Bool, save_start::Bool, save_end::Nothing, callback::CallbackSet{…}, dense::Bool, calck::Bool, dt::Float64, dtmin::Float64, dtmax::Float64, force_dtmin::Bool, adaptive::Bool, gamma::Rational{…}, abstol::Float64, reltol::Float64, qmin::Rational{…}, qmax::Int64, qsteady_min::Rational{…}, qsteady_max::Rational{…}, beta1::Nothing, beta2::Nothing, qoldinit::Rational{…}, controller::Nothing, fullnormalize::Bool, failfactor::Int64, maxiters::Int64, internalnorm::typeof(DiffEqBase.ODE_DEFAULT_NORM), internalopnorm::typeof(opnorm), isoutofdomain::typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), unstable_check::typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), verbose::Bool, timeseries_errors::Bool, dense_errors::Bool, advance_to_tstop::Bool, stop_at_next_tstop::Bool, initialize_save::Bool, progress::Bool, progress_steps::Int64, progress_name::String, progress_message::typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), progress_id::Symbol, userdata::Nothing, allow_extrapolation::Bool, initialize_integrator::Bool, alias::ODEAliasSpecifier, initializealg::OrdinaryDiffEqCore.DefaultInit, kwargs::@Kwargs{})
    @ OrdinaryDiffEqCore ~/.julia/packages/OrdinaryDiffEqCore/yWlnm/src/solve.jl:565
 [37] __init (repeats 5 times)
    @ ~/.julia/packages/OrdinaryDiffEqCore/yWlnm/src/solve.jl:11 [inlined]
 [38] __solve(::ODEProblem{…}, ::FBDF{…}; kwargs::@Kwargs{…})
    @ OrdinaryDiffEqCore ~/.julia/packages/OrdinaryDiffEqCore/yWlnm/src/solve.jl:6
 [39] __solve
    @ ~/.julia/packages/OrdinaryDiffEqCore/yWlnm/src/solve.jl:1 [inlined]
 [40] solve_call(_prob::ODEProblem{…}, args::FBDF{…}; merge_callbacks::Bool, kwargshandle::Nothing, kwargs::@Kwargs{…})
    @ DiffEqBase ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:635
 [41] solve_call
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:592 [inlined]
 [42] #solve_up#44
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:1167 [inlined]
 [43] solve_up
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:1145 [inlined]
 [44] #solve#42
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:1057 [inlined]
 [45] _adjoint_sensitivities(sol::ODESolution{…}, sensealg::QuadratureAdjoint{…}, alg::FBDF{…}; t::Vector{…}, dgdu_discrete::Function, dgdp_discrete::Nothing, dgdu_continuous::Nothing, dgdp_continuous::Nothing, g::Nothing, abstol::Float64, reltol::Float64, callback::Nothing, kwargs::@Kwargs{…})
    @ SciMLSensitivity ~/.julia/packages/SciMLSensitivity/hsM3z/src/quadrature_adjoint.jl:350
 [46] _adjoint_sensitivities
    @ ~/.julia/packages/SciMLSensitivity/hsM3z/src/quadrature_adjoint.jl:338 [inlined]
 [47] #adjoint_sensitivities#63
    @ ~/.julia/packages/SciMLSensitivity/hsM3z/src/sensitivity_interface.jl:401 [inlined]
 [48] (::SciMLSensitivity.var"#adjoint_sensitivity_backpass#323"{…})(Δ::ODESolution{…})
    @ SciMLSensitivity ~/.julia/packages/SciMLSensitivity/hsM3z/src/concrete_solve.jl:627
 [49] ZBack
    @ ~/.julia/packages/Zygote/HdT4O/src/compiler/chainrules.jl:222 [inlined]
 [50] (::Zygote.var"#kw_zpullback#58"{…})(dy::ODESolution{…})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/chainrules.jl:248
 [51] #305
    @ ~/.julia/packages/Zygote/HdT4O/src/lib/lib.jl:202 [inlined]
 [52] (::Zygote.var"#2189#back#307"{…})(Δ::ODESolution{…})
    @ Zygote ~/.julia/packages/ZygoteRules/CkVIK/src/adjoint.jl:72
 [53] #solve#42
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:1057 [inlined]
 [54] (::Zygote.Pullback{…})(Δ::ODESolution{…})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [55] #305
    @ ~/.julia/packages/Zygote/HdT4O/src/lib/lib.jl:202 [inlined]
 [56] (::Zygote.var"#2189#back#307"{…})(Δ::ODESolution{…})
    @ Zygote ~/.julia/packages/ZygoteRules/CkVIK/src/adjoint.jl:72
 [57] solve
    @ ~/.julia/packages/DiffEqBase/JuOYZ/src/solve.jl:1047 [inlined]
 [58] (::Zygote.Pullback{…})(Δ::ODESolution{…})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [59] predict
    @ ~/文档/julia/chromatography/Training the hybrid UDE-based model.jl:189 [inlined]
 [60] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::ChainRulesCore.Thunk{Lux.LossFunctionImpl.var"#2#5"{…}})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [61] pred
    @ ~/文档/julia/chromatography/Training the hybrid UDE-based model.jl:204 [inlined]
 [62] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::ChainRulesCore.Thunk{Lux.LossFunctionImpl.var"#2#5"{…}})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [63] loss
    @ ~/文档/julia/chromatography/Training the hybrid UDE-based model.jl:205 [inlined]
 [64] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [65] #22
    @ ~/文档/julia/chromatography/Training the hybrid UDE-based model.jl:208 [inlined]
 [66] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [67] (::Zygote.var"#88#89"{Zygote.Pullback{Tuple{…}, Tuple{…}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface.jl:97
 [68] withgradient(::Function, ::ComponentVector{Float64, Vector{Float64}, Tuple{Axis{…}}}, ::Vararg{Any})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface.jl:219
 [69] value_and_gradient
    @ ~/.julia/packages/DifferentiationInterface/Yk2Kt/ext/DifferentiationInterfaceZygoteExt/DifferentiationInterfaceZygoteExt.jl:115 [inlined]
 [70] value_and_gradient!(f::Function, grad::ComponentVector{…}, prep::DifferentiationInterface.NoGradientPrep{…}, backend::AutoZygote, x::ComponentVector{…}, contexts::DifferentiationInterface.Constant{…})
    @ DifferentiationInterfaceZygoteExt ~/.julia/packages/DifferentiationInterface/Yk2Kt/ext/DifferentiationInterfaceZygoteExt/DifferentiationInterfaceZygoteExt.jl:131
 [71] (::OptimizationZygoteExt.var"#fg!#16"{…})(res::ComponentVector{…}, θ::ComponentVector{…})
    @ OptimizationZygoteExt ~/.julia/packages/OptimizationBase/UXLhR/ext/OptimizationZygoteExt.jl:53
 [72] macro expansion
    @ ~/.julia/packages/OptimizationOptimisers/xC7Ic/src/OptimizationOptimisers.jl:101 [inlined]
 [73] macro expansion
    @ ~/.julia/packages/Optimization/PnmC0/src/utils.jl:32 [inlined]
 [74] __solve(cache::OptimizationCache{…})
    @ OptimizationOptimisers ~/.julia/packages/OptimizationOptimisers/xC7Ic/src/OptimizationOptimisers.jl:83
 [75] solve!(cache::OptimizationCache{…})
    @ SciMLBase ~/.julia/packages/SciMLBase/cwnDi/src/solve.jl:187
 [76] solve(::OptimizationProblem{…}, ::Adam{…}; kwargs::@Kwargs{…})
    @ SciMLBase ~/.julia/packages/SciMLBase/cwnDi/src/solve.jl:95
 [77] solve
    @ ~/.julia/packages/SciMLBase/cwnDi/src/solve.jl:92 [inlined]
 [78] train_one_round(p::ComponentVector{…}, t_train::Vector{…}, y_train::Vector{…}, opt::Adam{…}, maxiters::Int64)
    @ Main ~/文档/julia/chromatography/Training the hybrid UDE-based model.jl:210
 [79] macro expansion
    @ ./timing.jl:581 [inlined]
Some type information was truncated. Use `show(err)` to see complete types.

• seems like I have to const some variables and only differentiate with respecte to neural network parameteres when doing AD? I'm not familliar with Enzyme.jl, neither find similar tutorials like this one, which takes nn as part of DAE function, they are all the pure nn as the whole ODE function and train it. No case studies to referrence.

• The issue I mentioned above is about the usage of callback function, it's a small issue and I have figured it out, but Chris didn't resonse to the ERROR: Function argument passed to autodiff cannot be proven readonly