Merge branch 'main' into c++_20_support

Author: aliceb-nv

.coderabbit.yaml

@@ -15,13 +15,16 @@
 
 reviews:
   profile: chill
-  high_level_summary: false
+  high_level_summary: true
   auto_review:
-    enabled: false
+    enabled: true
     drafts: false
     ignore_usernames: ["rapids-bot", "GPUtester", "nv-automation-bot", "copy-pr-bot"]
   tools:
-    eslint:
+    markdownlint:
+      enabled: true
+    shellcheck:
       enabled: true
     gitleaks:
       enabled: true
+  sequence_diagrams: false

docs/cuopt/source/release-notes.rst RELEASE-NOTES.mddocs/cuopt/source/release-notes.rst renamed to RELEASE-NOTES.md

@@ -1,28 +1,21 @@
-=====================
-Release Notes
-=====================
+# Release Notes
 
-====================
-Release Notes 25.10
-====================
+## Release Notes 25.10
 
-New Features (25.10)
---------------------
+### New Features (25.10)
 
 - New barrier method for solving LPs. Uses cuDSS for sparse Cholesky / LDT.
 - Concurrent mode for LPs now uses PDLP, dual simplex, and barrier
 - New PDLP solver mode Stable3.
 - MIP presolve using Papilo (enabled by default). LP presolve using Papilo (optional).
 - Parallel branch and bound on the CPU: multiple best-first search and diving threads
 
-Breaking Changes (25.10)
-------------------------
+### Breaking Changes (25.10)
 
 - New PDLP Solver mode Stable3 is the default
 
 
-Improvements (25.10)
---------------------
+### Improvements (25.10)
 
 - Add setting "CUOPT_BARRIER_DUAL_INITIAL_POINT" to change the dual initial point used by barrier
 - CPUFJ for local search + simple rounding
@@ -60,8 +53,7 @@ Improvements (25.10)
 - This library now supports the QPS format, which is an extension of the standard MPS format for representing quadratic programming problems.
 
 
-Bug Fixes (25.10)
------------------
+### Bug Fixes (25.10)
 
 - Fix variables out of bounds caused by CPUFJ LP scratch thread
 - Fix the maybe-uninitialized compilation error
@@ -76,8 +68,7 @@ Bug Fixes (25.10)
 - Empty columns with infinite bounds are not removed
 
 
-Documentation (25.10)
----------------------
+### Documentation (25.10)
 
 - Add tutorial video links to Decompression
 - Add warmstart, model update, update docs
@@ -89,99 +80,86 @@ Documentation (25.10)
 - Doc update for container version update, and add ``nvidia-cuda-runtime`` as a dependency
 
 
-====================
-Release Notes 25.08
-====================
-
-
-New Features (25.08)
---------------------
-
-- Added Python API for LP and MILP (`#223 <https://github.com/NVIDIA/cuopt/pull/223>`_)
-
-Breaking Changes (25.08)
-------------------------
-
-- Fixed versioning for nightly and release package (`#175 <https://github.com/NVIDIA/cuopt/pull/175>`_)
-
-Improvements (25.08)
---------------------
-
-- New heuristic improvements (`#178 <https://github.com/NVIDIA/cuopt/pull/178>`_)
-- Add helm chart for cuOpt service (`#224 <https://github.com/NVIDIA/cuopt/pull/224>`_)
-- Add nightly container support (`#180 <https://github.com/NVIDIA/cuopt/pull/180>`_)
-- Adding deb package support as a beta feature (`#190 <https://github.com/NVIDIA/cuopt/pull/190>`_)
-- Use cusparsespmv_preprocess() now that Raft implements it (`#120 <https://github.com/NVIDIA/cuopt/pull/120>`_)
-- Create a bash script to run MPS files in parallel (`#87 <https://github.com/NVIDIA/cuopt/pull/87>`_)
-- Several fixes needed to compile cuOpt with LLVM (`#121 <https://github.com/NVIDIA/cuopt/pull/121>`_)
-- Small fixes for corner cases (`#130 <https://github.com/NVIDIA/cuopt/pull/130>`_)
-- Small improvements on how paths are handled in tests (`#129 <https://github.com/NVIDIA/cuopt/pull/129>`_)
-- Update cxxopts to v3.3.1 (`#128 <https://github.com/NVIDIA/cuopt/pull/128>`_)
-- Bump actions/checkout in nightly.yaml to v4 (`#230 <https://github.com/NVIDIA/cuopt/pull/230>`_)
-- Remove CUDA 11 specific changes from repo (`#222 <https://github.com/NVIDIA/cuopt/pull/222>`_)
-- Heuristic improvements with solution hash, MAB and simplex root solution (`#216 <https://github.com/NVIDIA/cuopt/pull/216>`_)
-- Various typos in comments and strings, note on result dir (`#200 <https://github.com/NVIDIA/cuopt/pull/200>`_)
-- Split very large tests into smaller individual test cases (`#152 <https://github.com/NVIDIA/cuopt/pull/152>`_)
-- Fix compile error when using clang with C++20 (`#145 <https://github.com/NVIDIA/cuopt/pull/145>`_)
-- Relax pinnings on several dependencies, remove nvidia channel (`#125 <https://github.com/NVIDIA/cuopt/pull/125>`_)
-- Fix compile error when building with clang (`#119 <https://github.com/NVIDIA/cuopt/pull/119>`_)
-- cuOpt service add healthcheck for / (`#114 <https://github.com/NVIDIA/cuopt/pull/114>`_)
-- refactor(shellcheck): fix all remaining shellcheck errors/warnings (`#99 <https://github.com/NVIDIA/cuopt/pull/99>`_)
-- Add CTK 12.9 fatbin flags to maintain existing binary sizes (`#58 <https://github.com/NVIDIA/cuopt/pull/58>`_)
-
-Bug Fixes (25.08)
------------------
-
-- Fixed a segfault on bnatt500 due to small mu leading to inf/nan (`#254 <https://github.com/NVIDIA/cuopt/pull/254>`_)
-- Fixed a bug in basis repair. Recover from numerical issues in primal update (`#249 <https://github.com/NVIDIA/cuopt/pull/249>`_)
-- Unset NDEBUG in cmake in assert mode (`#248 <https://github.com/NVIDIA/cuopt/pull/248>`_)
-- Manual cuda graph creation in load balanced bounds presolve (`#242 <https://github.com/NVIDIA/cuopt/pull/242>`_)
-- Fixed bug on initial solution size in the check and cuda set device order (`#226 <https://github.com/NVIDIA/cuopt/pull/226>`_)
-- Disable cuda graph in batched PDLP (`#225 <https://github.com/NVIDIA/cuopt/pull/225>`_)
-- Fix logging levels format with timestamps (`#201 <https://github.com/NVIDIA/cuopt/pull/201>`_)
-- Fix bug in scaling of dual slacks and sign of dual variables for >= constraints (`#191 <https://github.com/NVIDIA/cuopt/pull/191>`_)
-- Fix inversion crossover bug with PDP and prize collection (`#179 <https://github.com/NVIDIA/cuopt/pull/179>`_)
-- Fix a bug in extract_best_per_route kernel (`#156 <https://github.com/NVIDIA/cuopt/pull/156>`_)
-- Fix several bugs appeared in unit testing of JuMP interface (`#149 <https://github.com/NVIDIA/cuopt/pull/149>`_)
-- Fix incorrect reported solving time (`#131 <https://github.com/NVIDIA/cuopt/pull/131>`_)
-- Fix max offset (`#113 <https://github.com/NVIDIA/cuopt/pull/113>`_)
-- Fix batch graph capture issue caused by pinned memory allocator (`#110 <https://github.com/NVIDIA/cuopt/pull/110>`_)
-- Fix bug in optimization_problem_solution_t::copy_from (`#109 <https://github.com/NVIDIA/cuopt/pull/109>`_)
-- Fix issue when problem has an empty problem in PDLP (`#107 <https://github.com/NVIDIA/cuopt/pull/107>`_)
-- Fix crash on models with variables but no constraints (`#105 <https://github.com/NVIDIA/cuopt/pull/105>`_)
-- Fix inversion of constraint bounds in conditional bounds presolve (`#75 <https://github.com/NVIDIA/cuopt/pull/75>`_)
-- Fix data initialization in create depot node for max travel time feature (`#74 <https://github.com/NVIDIA/cuopt/pull/74>`_)
-
-Documentation (25.08)
----------------------
-
-- Added more pre-commit checks to ensure coding standards (`#213 <https://github.com/NVIDIA/cuopt/pull/213>`_)
-- Mention GAMS and GAMSPy in third-party modeling languages page in documentation (`#206 <https://github.com/NVIDIA/cuopt/pull/206>`_)
-- Enable doc build workflow and build script for PR and Nightly (`#203 <https://github.com/NVIDIA/cuopt/pull/203>`_)
-- Fix the link to Python docs in README (`#118 <https://github.com/NVIDIA/cuopt/pull/118>`_)
-- Add link checker for doc build and test (`#229 <https://github.com/NVIDIA/cuopt/pull/229>`_)
-
-====================
-Release Notes 25.05
-====================
-
-New Features (25.05)
---------------------
+## Release Notes 25.08
+
+### New Features (25.08)
+
+- Added Python API for LP and MILP ([#223](https://github.com/NVIDIA/cuopt/pull/223))
+
+### Breaking Changes (25.08)
+
+- Fixed versioning for nightly and release package ([#175](https://github.com/NVIDIA/cuopt/pull/175))
+
+### Improvements (25.08)
+
+- New heuristic improvements ([#178](https://github.com/NVIDIA/cuopt/pull/178))
+- Add helm chart for cuOpt service ([#224](https://github.com/NVIDIA/cuopt/pull/224))
+- Add nightly container support ([#180](https://github.com/NVIDIA/cuopt/pull/180))
+- Adding deb package support as a beta feature ([#190](https://github.com/NVIDIA/cuopt/pull/190))
+- Use cusparsespmv_preprocess() now that Raft implements it ([#120](https://github.com/NVIDIA/cuopt/pull/120))
+- Create a bash script to run MPS files in parallel ([#87](https://github.com/NVIDIA/cuopt/pull/87))
+- Several fixes needed to compile cuOpt with LLVM ([#121](https://github.com/NVIDIA/cuopt/pull/121))
+- Small fixes for corner cases ([#130](https://github.com/NVIDIA/cuopt/pull/130))
+- Small improvements on how paths are handled in tests ([#129](https://github.com/NVIDIA/cuopt/pull/129))
+- Update cxxopts to v3.3.1 ([#128](https://github.com/NVIDIA/cuopt/pull/128))
+- Bump actions/checkout in nightly.yaml to v4 ([#230](https://github.com/NVIDIA/cuopt/pull/230))
+- Remove CUDA 11 specific changes from repo ([#222](https://github.com/NVIDIA/cuopt/pull/222))
+- Heuristic improvements with solution hash, MAB and simplex root solution ([#216](https://github.com/NVIDIA/cuopt/pull/216))
+- Various typos in comments and strings, note on result dir ([#200](https://github.com/NVIDIA/cuopt/pull/200))
+- Split very large tests into smaller individual test cases ([#152](https://github.com/NVIDIA/cuopt/pull/152))
+- Fix compile error when using clang with C++20 ([#145](https://github.com/NVIDIA/cuopt/pull/145))
+- Relax pinnings on several dependencies, remove nvidia channel ([#125](https://github.com/NVIDIA/cuopt/pull/125))
+- Fix compile error when building with clang ([#119](https://github.com/NVIDIA/cuopt/pull/119))
+- cuOpt service add healthcheck for / ([#114](https://github.com/NVIDIA/cuopt/pull/114))
+- refactor(shellcheck): fix all remaining shellcheck errors/warnings ([#99](https://github.com/NVIDIA/cuopt/pull/99))
+- Add CTK 12.9 fatbin flags to maintain existing binary sizes ([#58](https://github.com/NVIDIA/cuopt/pull/58))
+
+### Bug Fixes (25.08)
+
+- Fixed a segfault on bnatt500 due to small mu leading to inf/nan ([#254](https://github.com/NVIDIA/cuopt/pull/254))
+- Fixed a bug in basis repair. Recover from numerical issues in primal update ([#249](https://github.com/NVIDIA/cuopt/pull/249))
+- Unset NDEBUG in cmake in assert mode ([#248](https://github.com/NVIDIA/cuopt/pull/248))
+- Manual cuda graph creation in load balanced bounds presolve ([#242](https://github.com/NVIDIA/cuopt/pull/242))
+- Fixed bug on initial solution size in the check and cuda set device order ([#226](https://github.com/NVIDIA/cuopt/pull/226))
+- Disable cuda graph in batched PDLP ([#225](https://github.com/NVIDIA/cuopt/pull/225))
+- Fix logging levels format with timestamps ([#201](https://github.com/NVIDIA/cuopt/pull/201))
+- Fix bug in scaling of dual slacks and sign of dual variables for >= constraints ([#191](https://github.com/NVIDIA/cuopt/pull/191))
+- Fix inversion crossover bug with PDP and prize collection ([#179](https://github.com/NVIDIA/cuopt/pull/179))
+- Fix a bug in extract_best_per_route kernel ([#156](https://github.com/NVIDIA/cuopt/pull/156))
+- Fix several bugs appeared in unit testing of JuMP interface ([#149](https://github.com/NVIDIA/cuopt/pull/149))
+- Fix incorrect reported solving time ([#131](https://github.com/NVIDIA/cuopt/pull/131))
+- Fix max offset ([#113](https://github.com/NVIDIA/cuopt/pull/113))
+- Fix batch graph capture issue caused by pinned memory allocator ([#110](https://github.com/NVIDIA/cuopt/pull/110))
+- Fix bug in optimization_problem_solution_t::copy_from ([#109](https://github.com/NVIDIA/cuopt/pull/109))
+- Fix issue when problem has an empty problem in PDLP ([#107](https://github.com/NVIDIA/cuopt/pull/107))
+- Fix crash on models with variables but no constraints ([#105](https://github.com/NVIDIA/cuopt/pull/105))
+- Fix inversion of constraint bounds in conditional bounds presolve ([#75](https://github.com/NVIDIA/cuopt/pull/75))
+- Fix data initialization in create depot node for max travel time feature ([#74](https://github.com/NVIDIA/cuopt/pull/74))
+
+### Documentation (25.08)
+
+- Added more pre-commit checks to ensure coding standards ([#213](https://github.com/NVIDIA/cuopt/pull/213))
+- Mention GAMS and GAMSPy in third-party modeling languages page in documentation ([#206](https://github.com/NVIDIA/cuopt/pull/206))
+- Enable doc build workflow and build script for PR and Nightly ([#203](https://github.com/NVIDIA/cuopt/pull/203))
+- Fix the link to Python docs in README ([#118](https://github.com/NVIDIA/cuopt/pull/118))
+- Add link checker for doc build and test ([#229](https://github.com/NVIDIA/cuopt/pull/229))
+
+## Release Notes 25.05
+
+### New Features (25.05)
 
 - Added concurrent mode that runs PDLP and Dual Simplex together
 - Added crossover from PDLP to Dual Simplex
 - Added a C API for LP and MILP
 - PDLP: Faster iterations and new more robust default PDLPSolverMode Stable2
 - Added support for writing out mps file containing user problem. Useful for debugging
 
-Breaking Changes (25.05)
-------------------------
+### Breaking Changes (25.05)
 
 - NoTermination is now a NumericalError
 - Split cuOpt as libcuopt and cuopt wheel
 
-Improvements (25.05)
---------------------
+### Improvements (25.05)
 
 - Hook up MILP Gap parameters and add info about number of nodes explored and simplex iterations
 - FJ bug fixes, tests and improvements
@@ -194,8 +172,7 @@ Improvements (25.05)
 - Logger file handling and bug fixes on MILP
 - Add shellcheck to pre-commit and fix warnings
 
-Bug Fixes (25.05)
------------------
+### Bug Fixes (25.05)
 
 - In the solution, ``termination_status`` should be cast to correct enum.
 - Fixed a bug using vehicle IDs in construct feasible solution algorithm.
@@ -218,6 +195,6 @@ Bug Fixes (25.05)
 - Fix obj constant on max. Fix undefined memory access at root
 - Allow long client version in service version check, this fixes the issue in case version is of the format 25.05.00.dev0
 
-Documentation (25.05)
----------------------
-- Restructure documementation to accomdate new APIs
+### Documentation (25.05)
+
+- Restructure documentation to accommodate new APIs

RELEASE_NOTES.md

@@ -693,7 +693,7 @@ def extract(file, dir, type):
         raise Exception(f"Unknown file extension found for extraction {file}")
     # download emps and compile
     # Disable emps for now
-    if type == "netlib":
+    if type == "netlib" and False:
         url = MittelmannInstances["emps"]
         file = os.path.join(dir, "emps.c")
         download(url, file)

benchmarks/linear_programming/utils/get_datasets.py

@@ -79,7 +79,6 @@ EXCLUDE_ARGS=(
   --exclude "librapids_logger.so"
   --exclude "libmps_parser.so"
   --exclude "librmm.so"
-  --exclude "libpapilo-core.a"
 )
 
 ci/build_wheel.sh libcuopt ${package_dir}

ci/build_wheel_libcuopt.sh

@@ -131,8 +131,6 @@ outputs:
       script:
         content: |
           cmake --install cpp/build
-          # Remove papilo _deps directory from cmake files
-          rm -rf "${PREFIX}/lib/cmake/papilo/_deps"
       dynamic_linking:
         overlinking_behavior: "error"
       prefix_detection:

conda/recipes/libcuopt/recipe.yaml

@@ -191,6 +191,7 @@ FetchContent_Declare(
   #we can switch to the main branch.
   GIT_TAG "741a2b9c8155b249d6df574d758b4d97d4417520"
   GIT_PROGRESS TRUE
+  EXCLUDE_FROM_ALL
   SYSTEM
 )
 

cpp/CMakeLists.txt

@@ -3575,6 +3575,9 @@ lp_status_t barrier_solver_t<i_t, f_t>::solve(f_t start_time,
   } catch (const raft::cuda_error& e) {
     settings.log.debug("Error in barrier_solver_t: %s\n", e.what());
     return lp_status_t::NUMERICAL_ISSUES;
+  } catch (const rmm::out_of_memory& e) {
+    settings.log.debug("Out of memory in barrier_solver_t: %s\n", e.what());
+    return lp_status_t::NUMERICAL_ISSUES;
   }
 }
 

cpp/src/dual_simplex/barrier.cu

@@ -28,6 +28,7 @@
 #include <mip/presolve/third_party_presolve.hpp>
 #include <mip/presolve/trivial_presolve.cuh>
 #include <mip/solver.cuh>
+#include <mip/utilities/sort_csr.cuh>
 
 #include <cuopt/linear_programming/pdlp/pdlp_hyper_params.cuh>
 #include <cuopt/linear_programming/pdlp/solver_settings.hpp>
@@ -444,7 +445,7 @@ optimization_problem_solution_t<i_t, f_t> run_barrier(
 {
   // Convert data structures to dual simplex format and back
   dual_simplex::user_problem_t<i_t, f_t> dual_simplex_problem =
-    cuopt_problem_to_simplex_problem<i_t, f_t>(problem);
+    cuopt_problem_to_simplex_problem<i_t, f_t>(problem.handle_ptr, problem);
   auto sol_dual_simplex = run_barrier(dual_simplex_problem, settings, timer);
   return convert_dual_simplex_sol(problem,
                                   std::get<0>(sol_dual_simplex),
@@ -515,7 +516,7 @@ optimization_problem_solution_t<i_t, f_t> run_dual_simplex(
 {
   // Convert data structures to dual simplex format and back
   dual_simplex::user_problem_t<i_t, f_t> dual_simplex_problem =
-    cuopt_problem_to_simplex_problem<i_t, f_t>(problem);
+    cuopt_problem_to_simplex_problem<i_t, f_t>(problem.handle_ptr, problem);
   auto sol_dual_simplex = run_dual_simplex(dual_simplex_problem, settings, timer);
   return convert_dual_simplex_sol(problem,
                                   std::get<0>(sol_dual_simplex),
@@ -671,16 +672,14 @@ optimization_problem_solution_t<i_t, f_t> run_concurrent(
   // Initialize the dual simplex structures before we run PDLP.
   // Otherwise, CUDA API calls to the problem stream may occur in both threads and throw graph
   // capture off
-  auto barrier_handle = raft::handle_t(*op_problem.get_handle_ptr());
-  detail::problem_t<i_t, f_t> d_barrier_problem(problem);
+  auto barrier_handle                  = raft::handle_t(*op_problem.get_handle_ptr());
   rmm::cuda_stream_view barrier_stream = rmm::cuda_stream_per_thread;
-  d_barrier_problem.handle_ptr         = &barrier_handle;
   raft::resource::set_cuda_stream(barrier_handle, barrier_stream);
   // Make sure allocations are done on the original stream
   problem.handle_ptr->sync_stream();
 
   dual_simplex::user_problem_t<i_t, f_t> dual_simplex_problem =
-    cuopt_problem_to_simplex_problem<i_t, f_t>(d_barrier_problem);
+    cuopt_problem_to_simplex_problem<i_t, f_t>(&barrier_handle, problem);
   // Create a thread for dual simplex
   std::unique_ptr<
     std::tuple<dual_simplex::lp_solution_t<i_t, f_t>, dual_simplex::lp_status_t, f_t, f_t, f_t>>
@@ -837,24 +836,24 @@ optimization_problem_solution_t<i_t, f_t> solve_lp(optimization_problem_t<i_t, f
     if (!run_presolve) { CUOPT_LOG_INFO("Third-party presolve is disabled, skipping"); }
 
     if (run_presolve) {
+      detail::sort_csr(op_problem);
       // allocate no more than 10% of the time limit to presolve.
       // Note that this is not the presolve time, but the time limit for presolve.
       // But no less than 1 second, to avoid early timeout triggering known crashes
       const double presolve_time_limit =
         std::max(1.0, std::min(0.1 * lp_timer.remaining_time(), 60.0));
-      presolver = std::make_unique<detail::third_party_presolve_t<i_t, f_t>>();
-      auto [reduced_problem, feasible] =
-        presolver->apply(op_problem,
-                         cuopt::linear_programming::problem_category_t::LP,
-                         settings.dual_postsolve,
-                         settings.tolerances.absolute_primal_tolerance,
-                         settings.tolerances.relative_primal_tolerance,
-                         presolve_time_limit);
-      if (!feasible) {
+      presolver   = std::make_unique<detail::third_party_presolve_t<i_t, f_t>>();
+      auto result = presolver->apply(op_problem,
+                                     cuopt::linear_programming::problem_category_t::LP,
+                                     settings.dual_postsolve,
+                                     settings.tolerances.absolute_primal_tolerance,
+                                     settings.tolerances.relative_primal_tolerance,
+                                     presolve_time_limit);
+      if (!result.has_value()) {
         return optimization_problem_solution_t<i_t, f_t>(
           pdlp_termination_status_t::PrimalInfeasible, op_problem.get_handle_ptr()->get_stream());
       }
-      problem       = detail::problem_t<i_t, f_t>(reduced_problem);
+      problem       = detail::problem_t<i_t, f_t>(result->reduced_problem);
       presolve_time = lp_timer.elapsed_time();
       CUOPT_LOG_INFO("Papilo presolve time: %f", presolve_time);
     }