Update documentation

Author: avik-pal

Project.toml

@@ -66,7 +66,7 @@ NonlinearProblemLibrary = "0.1"
 Pkg = "1"
 PrecompileTools = "1"
 Printf = "<0.0.1, 1"
-Random = "1"
+Random = "<0.0.1, 1"
 RecursiveArrayTools = "2"
 Reexport = "0.2, 1"
 SafeTestsets = "0.1"

docs/Project.toml

@@ -9,6 +9,7 @@ LinearSolve = "7ed4a6bd-45f5-4d41-b270-4a48e9bafcae"
 ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78"
 NonlinearSolve = "8913a72c-1f9b-4ce2-8d82-65094dcecaec"
 NonlinearSolveMINPACK = "c100e077-885d-495a-a2ea-599e143bf69d"
+Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 SciMLBase = "0bca4576-84f4-4d90-8ffe-ffa030f20462"
 SciMLNLSolve = "e9a6253c-8580-4d32-9898-8661bb511710"
 SimpleNonlinearSolve = "727e6d20-b764-4bd8-a329-72de5adea6c7"
@@ -26,12 +27,13 @@ Documenter = "1"
 IncompleteLU = "0.2"
 LinearSolve = "2"
 ModelingToolkit = "8"
-NonlinearSolve = "1, 2"
+NonlinearSolve = "3"
 NonlinearSolveMINPACK = "0.1"
+Random = "<0.0.1, 1"
 SciMLBase = "2.4"
 SciMLNLSolve = "0.1"
 SimpleNonlinearSolve = "1"
 StaticArrays = "1"
-SteadyStateDiffEq = "1.10, 2"
+SteadyStateDiffEq = "2"
 Sundials = "4.11"
 Symbolics = "4, 5"

docs/src/api/fastlevenbergmarquardt.md

@@ -0,0 +1,19 @@
+# FastLevenbergMarquardt.jl
+
+This is a wrapper package for importing solvers from FastLevenbergMarquardt.jl into the
+SciML interface. Note that these solvers do not come by default, and thus one needs to
+install the package before using these solvers:
+
+```julia
+using Pkg
+Pkg.add("FastLevenbergMarquardt")
+using FastLevenbergMarquardt
+```
+
+These methods can be used independently of the rest of NonlinearSolve.jl
+
+## Solver API
+
+```@docs
+FastLevenbergMarquardtJL
+```

docs/src/api/leastsquaresoptim.md

@@ -0,0 +1,19 @@
+# LeastSquaresOptim.jl
+
+This is a wrapper package for importing solvers from LeastSquaresOptim.jl into the SciML
+interface. Note that these solvers do not come by default, and thus one needs to install
+the package before using these solvers:
+
+```julia
+using Pkg
+Pkg.add("LeastSquaresOptim")
+using LeastSquaresOptim
+```
+
+These methods can be used independently of the rest of NonlinearSolve.jl
+
+## Solver API
+
+```@docs
+LeastSquaresOptimJL
+```

docs/src/api/minpack.md

@@ -1,8 +1,8 @@
 # MINPACK.jl
 
 This is a wrapper package for importing solvers from Sundials into the SciML interface.
-Note that these solvers do not come by default, and thus one needs to install
-the package before using these solvers:
+Note that these solvers do not come by default, and thus one needs to install the package
+before using these solvers:
 
 ```julia
 using Pkg

docs/src/api/nlsolve.md

@@ -1,8 +1,8 @@
 # NLsolve.jl
 
 This is a wrapper package for importing solvers from NLsolve.jl into the SciML interface.
-Note that these solvers do not come by default, and thus one needs to install
-the package before using these solvers:
+Note that these solvers do not come by default, and thus one needs to install the package
+before using these solvers:
 
 ```julia
 using Pkg

docs/src/api/simplenonlinearsolve.md

@@ -6,7 +6,8 @@ These methods can be used independently of the rest of NonlinearSolve.jl
 
 ### Interval Methods
 
-These methods are suited for interval (scalar) root-finding problems, i.e. `IntervalNonlinearProblem`.
+These methods are suited for interval (scalar) root-finding problems,
+i.e. `IntervalNonlinearProblem`.
 
 ```@docs
 ITP
@@ -18,14 +19,15 @@ Brent
 
 ### General Methods
 
-These methods are suited for any general nonlinear root-finding problem, i.e. `NonlinearProblem`.
+These methods are suited for any general nonlinear root-finding problem, i.e.
+`NonlinearProblem`.
 
 ```@docs
 SimpleNewtonRaphson
-Broyden
+SimpleBroyden
 SimpleHalley
-Klement
+SimpleKlement
 SimpleTrustRegion
 SimpleDFSane
-LBroyden
+SimpleLimitedMemoryBroyden
 ```

docs/src/api/steadystatediffeq.md

@@ -1,8 +1,8 @@
 # SteadyStateDiffEq.jl
 
 This is a wrapper package for using ODE solvers from
-[DifferentialEquations.jl](https://docs.sciml.ai/DiffEqDocs/stable/) into the SciML interface.
-Note that these solvers do not come by default, and thus one needs to install
+[DifferentialEquations.jl](https://docs.sciml.ai/DiffEqDocs/stable/) into the SciML
+interface. Note that these solvers do not come by default, and thus one needs to install
 the package before using these solvers:
 
 ```julia

docs/src/api/sundials.md

@@ -1,8 +1,8 @@
 # Sundials.jl
 
 This is a wrapper package for importing solvers from Sundials into the SciML interface.
-Note that these solvers do not come by default, and thus one needs to install
-the package before using these solvers:
+Note that these solvers do not come by default, and thus one needs to install the package
+before using these solvers:
 
 ```julia
 using Pkg

docs/src/basics/FAQ.md

@@ -8,7 +8,7 @@ On the test example:
 ```@example
 using NonlinearSolve, BenchmarkTools
 
-N = 100_000;
+const N = 100_000;
 levels = 1.5 .* rand(N);
 out = zeros(N);
 myfun(x, lv) = x * sin(x) - lv
@@ -31,8 +31,62 @@ end
 @btime f2(out, levels, 1.0)
 ```
 
-MATLAB 2022a achieves 1.66s. Try this code yourself: we receive 0.06 seconds, or a 28x speedup.
-This example is still not optimized in the Julia code, and we expect an improvement in a near
-future version.
+MATLAB 2022a achieves 1.66s. Try this code yourself: we receive 0.009 seconds, or a 184x
+speedup.
 
 For more information on performance of SciML, see the [SciMLBenchmarks](https://docs.sciml.ai/SciMLBenchmarksOutput/stable/).
+
+## The solver tried to set a Dual Number in my Vector of Floats.How do I fix that?
+
+This is a common problem that occurs if the code was not written to be generic based on the
+input types. For example, consider this example taken from
+[this issue](https://github.com/SciML/NonlinearSolve.jl/issues/298)
+
+```@example dual_error_faq
+using NonlinearSolve, Random
+
+function fff_incorrect(var, p)
+    v_true = [1.0, 0.1, 2.0, 0.5]
+    xx = [1.0, 2.0, 3.0, 4.0]
+    xx[1] = var[1] - v_true[1]
+    return var - v_true
+end
+
+v_true = [1.0, 0.1, 2.0, 0.5]
+v_init = v_true .+ randn!(similar(v_true)) * 0.1
+
+prob_oop = NonlinearLeastSquaresProblem{false}(fff_incorrect, v_init)
+try
+    sol = solve(prob_oop, LevenbergMarquardt(); maxiters = 10000, abstol = 1e-8)
+catch e
+    @error e
+end
+```
+
+Essentially what happened was, NonlinearSolve checked that we can use ForwardDiff.jl to
+differentiate the function based on the input types. However, this function has
+`xx = [1.0, 2.0, 3.0, 4.0]` followed by a `xx[1] = var[1] - v_true[1]` where `var` might
+be a Dual number. This causes the error. To fix it:
+
+ 1. Specify the `autodiff` to be `AutoFiniteDiff`
+
+```@example dual_error_faq
+sol = solve(prob_oop, LevenbergMarquardt(; autodiff = AutoFiniteDiff()); maxiters = 10000,
+    abstol = 1e-8)
+```
+
+This worked but, Finite Differencing is not the recommended approach in any scenario.
+Instead, rewrite the function to use
+[PreallocationTools.jl](https://github.com/SciML/PreallocationTools.jl) or write it as
+
+```@example dual_error_faq
+function fff_correct(var, p)
+    v_true = [1.0, 0.1, 2.0, 0.5]
+    xx = eltype(var)[1.0, 2.0, 3.0, 4.0]
+    xx[1] = var[1] - v_true[1]
+    return xx - v_true
+end
+
+prob_oop = NonlinearLeastSquaresProblem{false}(fff_correct, v_init)
+sol = solve(prob_oop, LevenbergMarquardt(); maxiters = 10000, abstol = 1e-8)
+```