Merge pull request #2600 from ERGO-Code/latest-docs

Documentation updates

Author: jajhall

docs/Project.toml

@@ -3,5 +3,5 @@ Clang = "40e3b903-d033-50b4-a0cc-940c62c95e31"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
 
 [compat]
-Clang = "0.14, 0.17, 0.18"
+#Clang = "0.14, 0.17, 0.18"
 Documenter = "0.27"

docs/README.md

@@ -57,7 +57,8 @@ Documenter.makedocs(
             "guide/further.md",
             "guide/advanced.md",
             "guide/gpu.md",
-            "guide/kkt.md"
+            "guide/kkt.md",
+            "guide/numerics.md"
         ],
 	"Data structures" => Any[
 	    "structures/index.md",

docs/make.jl

@@ -1,22 +1,49 @@
 # [GPU acceleration](@id gpu)
 
-From HiGHS v1.10.0, its first order primal-dual LP (PDLP) solver [cuPDLP-C](https://github.com/COPT-Public/cuPDLP-C) can be run on an NVIDIA GPU under Linux and Windows. However, to achieve this, CUDA utilities must be installed and HiGHS must be built locally using CMake, as described below.
+From HiGHS v1.10.0, its first order primal-dual LP (PDLP) solver
+[cuPDLP-C](https://github.com/COPT-Public/cuPDLP-C) can be run on an
+NVIDIA GPU under Linux and Windows. However, to achieve this, CUDA
+utilities must be installed and HiGHS must be built locally using
+CMake, as described below.
 
 ### PDLP: A health warning
 
-First order solvers for LP are still very much "work in progress". Although impressive results have been reported, these are often to lower accuracy than is achieved by simplex and interior point solvers, have been obtained using top-of-the-range GPUs, and not achieved for all problem classes. Note that, due to PDLP using relative termination conditions, a solution deemed optimal by PDLP may not be accepted as optimal by HiGHS. The user should consider the infeasibility data returned by [HighsInfo](@ref HighsInfo) to decide whether the solution is acceptable to them.
+First order solvers for LP are still very much "work in
+progress". Although impressive results have been reported, these are
+often to lower accuracy than is achieved by simplex and interior point
+solvers, have been obtained using top-of-the-range GPUs, and not
+achieved for all problem classes. Note that, due to PDLP using
+relative termination conditions, a solution deemed optimal by PDLP may
+not be accepted as optimal by HiGHS. The user should consider the
+infeasibility data returned by [HighsInfo](@ref HighsInfo) to decide
+whether the solution is acceptable to them.
 
 #### Termination criteria
 
-Although the PDLP solver may report that it has terminated with an optimal solution, HiGHS may identify that the solution returned by PDLP is not optimal. As discussed in [HiGHS feasibilty and optimality tolerances](@ref kkt), this is due to PDLP using relative termination criteria. 
-
-If you use the HiGHS PDLP solver, in the first instance it is recommended that you increase the feasibility and optimality tolerances to `1e-4`, since this will result in the algorithm terminating much sooner. There are multiple feasibility and optimality tolerances, but all will be set to the value of the [`kkt_tolerance`](@ref option-kkt-tolerance) option (if it differs from its default value of `1e-4`) so this is recommended in the first instance. 
+Although the PDLP solver may report that it has terminated with an
+optimal solution, HiGHS may identify that the solution returned by
+PDLP is not optimal. As discussed in [HiGHS feasibility and optimality
+tolerances](@ref kkt), this is due to PDLP using relative termination
+criteria and (unlike interior point solvers) not satisfying
+feasibility to high accuracy.
+
+If you use the HiGHS PDLP solver, in the first instance it is
+recommended that you increase the feasibility and optimality
+tolerances to `1e-4`, since this will result in the algorithm
+terminating much sooner. There are multiple feasibility and optimality
+tolerances, but all will be set to the value of the
+[`kkt_tolerance`](@ref option-kkt-tolerance) option (if it differs
+from its default value of `1e-4`) so this is recommended in the first
+instance.
 
 ### Requirements
 
 CUDA Toolkit and CMake. 
 
-A [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) installation is required, along with the matching NVIDIA driver. Please install both following the instructions on NVIDIA's website.
+A [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit)
+installation is required, along with the matching NVIDIA
+driver. Please install both following the instructions on NVIDIA's
+website.
 
 HiGHS must be build locally with CMake. 
 

docs/src/guide/gpu.md

@@ -1,6 +1,6 @@
-# [Feasibilty and optimality](@id kkt)
+# [Feasibility and optimality](@id kkt)
 
-Mathematically, continuous optimization problems have exact feasibilty
+Mathematically, continuous optimization problems have exact feasibility
 and optimality conditions. However, since solvers cannot always
 satisfy these conditions exactly when using floating-point arithmetic,
 they do so to within tolerances. As explored below, some solvers aim
@@ -12,7 +12,7 @@ and can give a misleading claim of optimality. To achieve consistency,
 HiGHS reassesses the optimal solution claimed by such a solver in a
 reasonable and uniform manner.
 
-### Feasibilty and optimality conditions
+### Feasibility and optimality conditions
 
 To discuss tolerances and their use in different solvers, consider the
 standard form linear programming (LP) problem with ``n`` variables and
@@ -26,7 +26,7 @@ standard form linear programming (LP) problem with ``n`` variables and
 \end{aligned}
 ```
 
-The feasibilty and optimality conditions (KKT conditions) are that, at
+The feasibility and optimality conditions (KKT conditions) are that, at
 a point ``x``, there exist (row) dual values ``y`` and reduced costs
 (column dual values) ``s`` such that
 
@@ -113,11 +113,12 @@ tolerance.
 
 ### Solutions without a corresponding basis
 
-The HiGHS PDLP solver and the interior point solver without crossover
-(IPX) yield "optimal" primal and dual values that satisfy internal
-conditions for termination of the underlying algorithm. These
-conditions are discussed below, and are used for good reason. However
-they can lead to a misleading claim of optimality.
+The HiGHS PDLP solver and the interior point solvers (HiPO and IPX)
+without crossover yield "optimal" primal and dual values that satisfy
+internal conditions for termination of the underlying algorithm. These
+conditions are discussed below, and are used for good reason. However,
+particularly in the case of PDLP, they can lead to a misleading claim
+of optimality.
 
 #### Interior point solutions
 
@@ -138,6 +139,13 @@ HiGHS. It terminates when
 \end{aligned}
 ```
 
+In practice, the feasibility tolerances are readily satisfied when
+using interior point solvers. Hence their termination generally
+depends on satisfying the optimality tolerance, with relative
+feasibility satisfied to a higher tolerance than required. Thus the
+solution obtained using interior point solvers typically has small
+absolute feasibility errors.
+
 #### PDLP solutions
 
 The PDLP algorithm uses an independent [optimality tolerance](@ref
@@ -155,27 +163,32 @@ feasibility by construction. It terminates when
 \end{aligned}
 ```
 
+Unlike interior point solvers, PDLP does not readily satisfy the
+relative feasibility tolerances. Although they and the optimality
+tolerance are satisfied on successful termination, the solution
+obtained may have meaningful absolute feasibility errors.
+
 #### HiGHS solutions
 
-The relative measures used by PDLP and IPX assume that all components
-of the cost and RHS vectors are relevant. When an LP problem is in
-standard form this is true for ``b``, but not necessarily for the cost
-vector ``c``. Consider a large component of ``c`` for which the
-corresponding reduced cost value in ``s`` is also large, in which case
-the LP solution is insensitive to the cost. This component will
-contribute significantly to ``\|c\|`` and, hence, the RHS of the dual
-residual condition, allowing large values of ``\|c-A^Ty-s\|`` to be
-accepted. However, this can lead to unacceptably large absolute
-residual errors and non-optimal solutions being deemed "optimal". When
-equations in ``Ax=b`` correspond to inequality constraints with large
-RHS values and a slack variable (so the constraint is redundant) the
-same issue occurs in the case of primal residual errors. The solution
-of the LP is not sensitive to this large RHS value, but its
-contribution to ``||b||`` can allow large absolute primal residual
-errors to be overlooked.
+The relative measures used by HiPO, IPX and PDLP assume that all
+components of the cost and RHS vectors are relevant. When an LP
+problem is in standard form this is true for ``b``, but not
+necessarily for the cost vector ``c``. Consider a large component of
+``c`` for which the corresponding reduced cost value in ``s`` is also
+large, in which case the LP solution is insensitive to the cost. This
+component will contribute significantly to ``\|c\|`` and, hence, the
+RHS of the dual residual condition, allowing large values of
+``\|c-A^Ty-s\|`` to be accepted. However, this can lead to
+unacceptably large absolute residual errors and non-optimal solutions
+being deemed "optimal". When equations in ``Ax=b`` correspond to
+inequality constraints with large RHS values and a slack variable (so
+the constraint is redundant) the same issue occurs in the case of
+primal residual errors. The solution of the LP is not sensitive to
+this large RHS value, but its contribution to ``||b||`` can allow
+large absolute primal residual errors to be overlooked.
 
 To make an informed assessment of whether an "optimal" solution
-obtained by IPX or PDLP is acceptable, HiGHS computes infinity norm
+obtained by HiPO, IPX or PDLP is acceptable, HiGHS computes infinity norm
 measures of ``b`` and ``c`` corresponding to the components that
 define the optimal solution. For ``c`` these are the components
 corresponding to positive values of ``x`` and reduced costs that are

docs/src/guide/kkt.md

@@ -0,0 +1,55 @@
+# [Numerical considerations](@id numerics)
+
+Optimization solvers cannot be expected to find the exact solution of
+a problem, since it may not be possible to represent that solution
+using floating-point arithemtic. However, solvers will typically run
+faster and find more accurate solutions if the problem has good
+numerical properties. Ideally the optimal value of all primal
+variables (and dual variables when relevant) will be of order
+unity. This typically occurs if all objective and constraint matrix
+coefficients, as well as finite variable and constraint bounds, are of
+order unity. Whilst there may be some reasons why this ideal cannot be
+achieved in all models, there are many pitfalls to avoid. For an
+insight into reasons why a model may have bad numerical properties and
+how to avoid them, users are recommended to study this [JuMP
+tutorial](https://jump.dev/JuMP.jl/stable/tutorials/getting_started/tolerances/). Improving
+the numerical properties of a model will typically lead to it being
+solved faster and more accurately/reliably, so the investment should
+pay off!
+
+Internally, the HiGHS continuous optimization solvers scale the
+constraint matrix to improve the numerical properties of the problem,
+[feasibility and optimality tolerances](@ref kkt) are determined with
+respect to the original, unscaled problem. However, faced with a model
+with bad numerical properties, there is only so much that HiGHS can do
+to solve it efficiently and accurately.
+
+If the optimal values of many variables in a model are typically very
+large, this can correspond to very large values of the objective
+coefficients and finite variable and constraint bounds. Since most of
+the HiGHS solvers terminate according to small absolute [feasibility
+tolerances](@ref kkt), large objective coefficients and bounds force
+the solvers to achieve an accuracy that may be unrealsitic in the
+context of a model. As well as having an impact on efficiency, the
+solver may ultimately be unable to achieve the required accuracy and
+fail. Objective coefficients and bounds that are less than the
+feasibility and optimality tolerances can also be problematic,
+although this is less common and less serious.
+
+HiGHS offers a facility to enable users to assess the consequences of
+better problem scaling, in cases where some objective coefficients or
+bounds are large, or if all objective coefficients or bounds are
+small. By setting the options [__user\_objective\_scale__](@ref
+option-user_objective_scale) and/or [__user\_bound\_scale__](@ref
+option-user_bound_scale), HiGHS will solve the given model with
+uniform scaling of the objective coefficients or bounds. Note that
+these options define the exponent in power-of-two scaling factors so
+that model accuracy is not compromised. After solving the problem,
+feasibility and optimality will be assessed for the original model,
+with a warning given if the tolerances are not satisfied. Note that
+uniform scaling of bounds on discrete variables is not possible, and
+is achieved implicitly by scaling their cost and matrix
+coefficients. Also, when bounds on variables in a quadratic
+programming problem are scaled up (down), the values in the Hessian
+matrix must be scaled down (up) so that the overall scaling of the
+objective is uniform.

docs/src/guide/numerics.md

@@ -115,13 +115,13 @@
 - Range: {0, 2147483647}
 - Default: 0
 
-## user\_objective\_scale
+## [user\_objective\_scale](@id option-user_objective_scale)
 - Exponent of power-of-two objective scaling for model
 - Type: integer
 - Range: {-2147483647, 2147483647}
 - Default: 0
 
-## user\_bound\_scale
+## [user\_bound\_scale](@id option-user_bound_scale)
 - Exponent of power-of-two bound scaling for model
 - Type: integer
 - Range: {-2147483647, 2147483647}

docs/src/options/definitions.md

@@ -1727,7 +1727,7 @@ bool reportKktFailures(const HighsLp& lp, const HighsOptions& options,
     optimality_tolerance = options.kkt_tolerance;
   }
 
-  const bool force_report = false;
+  const bool force_report = options.log_dev_level >= kHighsLogDevLevelInfo;
   const bool complementarity_error =
       !is_mip && info.primal_dual_objective_error > optimality_tolerance;
   const bool integrality_error =

highs/lp_data/HighsSolution.cpp

@@ -230,7 +230,8 @@ HighsStatus solveLpCupdlp(const HighsOptions& options, HighsTimer& timer,
       assert(111 == 777);
     }
 #if CUPDLP_DEBUG
-    analysePdlpSolution(options, lp, highs_solution);
+    if (options.log_dev_level >= kHighsLogDevLevelDetailed)
+      analysePdlpSolution(options, lp, highs_solution);
 #endif
   } else {
     // Failure return from LP_SolvePDHG