Skip to content

[BUG] Segfault in mip_solve.run_small_tests #668

@chris-maes

Description

@chris-maes

On PR 500 (which should be unreleated to MIP) we see a segfault in the mip_solve.run_small_tests

https://github.com/NVIDIA/cuopt/actions/runs/20034144618/job/57454389819?pr=500

[ RUN      ] mip_solve.run_small_tests
cuOpt version: 25.12.0, git hash: , host arch: x86_64, device archs: 75-real,80-real,86-real,90a-real,100f-real,120a-real,120
CPU: AMD EPYC 7313P 16-Core Processor, threads (physical/logical): 12/12, RAM: 42.42 GiB
CUDA 13.0, device: NVIDIA H100 PCIe (ID 0), VRAM: 79.18 GiB
CUDA device UUID: 7dffffff85ffffffe403-78ffffffd8-ffff

Solving a problem with 233 constraints, 2013 variables (1647 integers), and 2745 nonzeros
Problem scaling:
Objective coefficents range:          [2e+00, 7e+02]
Constraint matrix coefficients range: [1e+00, 2e+02]
Constraint rhs / bounds range:        [1e+01, 1e+03]
Variable bounds range:                [0e+00, 2e+02]

Original problem: 233 constraints, 2013 variables, 2745 nonzeros
Calling Papilo presolver
Presolve status: reduced the problem
Presolve removed: 0 constraints, 0 variables, 0 nonzeros
Presolved problem: 233 constraints, 2013 variables, 2745 nonzeros
Papilo presolve time: 0.072459
Objective offset 0.000000 scaling_factor 1.000000
Running presolve!
After trivial presolve: 233 constraints, 2013 variables, objective offset 0.000000.
Using 11 CPU threads for B&B
Solving LP root relaxation
Scaling matrix. Maximum column norm 1.000000e+00, minimum column norm 1.598131e-02
Dual Simplex Phase 1
Dual feasible solution found.
Dual Simplex Phase 2
 Iter     Objective           Num Inf.  Sum Inf.     Perturb  Time
    1 +0.0000000000000000e+00      26 8.74174830e+04 0.00e+00 0.00
Removed perturbation of 2.91e-06.


Root relaxation solution found in 317 iterations and 0.02s
Root relaxation objective +2.87906569e+03


Strong branching using 11 threads and 29 fractional variables
Exploring the B&B tree using 11 threads (best-first = 2, diving = 9)
 | Explored | Unexplored |    Objective    |     Bound     | Depth | Iter/Node |   Gap    |  Time  |
D       105          107    +6.801190e+03    +2.911428e+03       61   1.6e+01     57.2%      0.10
D       105          107    +6.775710e+03    +2.911428e+03       62   1.6e+01     57.0%      0.10
D       105          107    +6.758290e+03    +2.911428e+03       63   1.6e+01     56.9%      0.10
D       105          107    +6.748930e+03    +2.911428e+03       64   1.6e+01     56.9%      0.10
D       106          108    +6.740610e+03    +2.911428e+03       65   1.6e+01     56.8%      0.10
D       106          108    +6.736970e+03    +2.911428e+03       66   1.6e+01     56.8%      0.10
D       107          109    +6.732290e+03    +2.911428e+03       67   1.6e+01     56.8%      0.10
D       110          112    +6.728390e+03    +2.911428e+03       68   1.7e+01     56.7%      0.11
D       115          117    +6.346150e+03    +2.911428e+03       60   1.7e+01     54.1%      0.11
D       116          118    +6.305950e+03    +2.911428e+03       61   1.7e+01     53.8%      0.11
D       116          118    +6.284350e+03    +2.911428e+03       62   1.7e+01     53.7%      0.11
D       116          118    +6.265150e+03    +2.911428e+03       63   1.7e+01     53.5%      0.11
D       117          119    +6.256750e+03    +2.911428e+03       65   1.7e+01     53.5%      0.11
D       118          120    +6.245950e+03    +2.911428e+03       69   1.7e+01     53.4%      0.12
D       120          122    +6.098080e+03    +2.911428e+03       66   1.8e+01     52.3%      0.12
D       127          129    +5.820450e+03    +2.911428e+03       49   1.8e+01     50.0%      0.12
D       140          142    +5.745970e+03    +2.911428e+03       50   1.7e+01     49.3%      0.13
D       145          147    +5.736050e+03    +2.911428e+03       52   1.8e+01     49.2%      0.13
D       149          151    +5.727090e+03    +2.911428e+03       53   1.8e+01     49.2%      0.14
D       154          156    +5.723170e+03    +2.911428e+03       54   1.9e+01     49.1%      0.14
D       164          158    +5.718130e+03    +2.911428e+03       55   1.8e+01     49.1%      0.14
D       186          171    +5.712540e+03    +2.914722e+03       56   1.6e+01     49.0%      0.15
D       200          182    +5.708620e+03    +2.916072e+03       57   1.5e+01     48.9%      0.15
D       201          183    +5.703580e+03    +2.916072e+03       58   1.5e+01     48.9%      0.15
D       209          191    +5.693210e+03    +2.916072e+03       52   1.5e+01     48.8%      0.16
D       215          197    +5.683130e+03    +2.916072e+03       53   1.6e+01     48.7%      0.16
D       222          204    +5.674170e+03    +2.916072e+03       54   1.6e+01     48.6%      0.16
D       227          209    +5.670250e+03    +2.916072e+03       55   1.6e+01     48.6%      0.17
D       228          210    +5.665210e+03    +2.916072e+03       56   1.6e+01     48.5%      0.17
D       247          229    +5.661040e+03    +2.916072e+03       60   1.5e+01     48.5%      0.18
D       247          229    +5.657120e+03    +2.916072e+03       61   1.5e+01     48.5%      0.18
D       247          229    +5.652080e+03    +2.916072e+03       62   1.5e+01     48.4%      0.18
D       248          230    +5.646950e+03    +2.916072e+03       54   1.5e+01     48.4%      0.18
D       248          230    +5.637990e+03    +2.916072e+03       55   1.5e+01     48.3%      0.18
D       249          231    +5.634070e+03    +2.916072e+03       56   1.6e+01     48.2%      0.18
D       251          233    +5.629030e+03    +2.916072e+03       57   1.6e+01     48.2%      0.18
D       411          345    +5.621540e+03    +2.920172e+03       97   1.5e+01     48.1%      0.26
D       423          357    +5.618550e+03    +2.920172e+03       56   1.5e+01     48.0%      0.26
D       423          357    +5.614630e+03    +2.920172e+03       57   1.5e+01     48.0%      0.26
D       426          360    +5.609590e+03    +2.920172e+03       58   1.5e+01     47.9%      0.27
D       703          467    +5.601270e+03    +2.922138e+03       57   1.3e+01     47.8%      0.34
D       705          469    +5.597350e+03    +2.922138e+03       58   1.4e+01     47.8%      0.35
D       710          474    +5.592310e+03    +2.923029e+03       59   1.3e+01     47.7%      0.35
D       757          505    +5.590720e+03    +2.924276e+03       55   1.3e+01     47.7%      0.36
D       776          524    +5.583720e+03    +2.924276e+03       57   1.3e+01     47.6%      0.37
D       796          544    +5.574720e+03    +2.924276e+03       61   1.3e+01     47.5%      0.38
D       906          622    +5.567220e+03    +2.927622e+03       95   1.4e+01     47.4%      0.42
D       956          658    +5.564510e+03    +2.930747e+03       59   1.4e+01     47.3%      0.44
D       993          685    +5.555960e+03    +2.930747e+03       62   1.4e+01     47.3%      0.45
D      1013          695    +5.552270e+03    +2.930936e+03       59   1.4e+01     47.2%      0.46
D      1017          697    +5.547230e+03    +2.930936e+03       60   1.4e+01     47.2%      0.46
D      1164          786    +5.545880e+03    +2.934280e+03       63   1.4e+01     47.1%      0.52
D      1215          815    +5.544710e+03    +2.936347e+03       60   1.4e+01     47.0%      0.54
D      1216          816    +5.539670e+03    +2.936347e+03       61   1.4e+01     47.0%      0.55
D      1239          823    +5.538380e+03    +2.936347e+03       97   1.4e+01     47.0%      0.55
D      1297          873    +5.536920e+03    +2.939420e+03       64   1.4e+01     46.9%      0.58
D      1333          895    +5.534990e+03    +2.939935e+03       61   1.4e+01     46.9%      0.60
D      1335          897    +5.529950e+03    +2.939935e+03       62   1.4e+01     46.8%      0.60
D      1343          905    +5.529420e+03    +2.939935e+03       98   1.4e+01     46.8%      0.61
D      1404          930    +5.525500e+03    +2.940540e+03       99   1.4e+01     46.8%      0.64
D      1553         1043    +5.520460e+03    +2.943550e+03      100   1.4e+01     46.7%      0.70
D      1566         1056    +5.516410e+03    +2.943550e+03       61   1.4e+01     46.6%      0.71
D      1831         1237    +5.513890e+03    +2.949410e+03       61   1.4e+01     46.5%      0.84
D      1832         1238    +5.508850e+03    +2.949410e+03       62   1.4e+01     46.5%      0.84
D      2002         1318    +5.504240e+03    +2.950100e+03       96   1.4e+01     46.4%      0.92
D      2012         1328    +5.504170e+03    +2.950100e+03       62   1.4e+01     46.4%      0.93
D      2014         1330    +5.499130e+03    +2.950100e+03       63   1.4e+01     46.4%      0.93
D      2046         1362    +5.492980e+03    +2.950100e+03       63   1.4e+01     46.3%      0.95
D      2161         1419    +5.485480e+03    +2.950925e+03       97   1.4e+01     46.2%      1.00
D      2173         1431    +5.482570e+03    +2.950925e+03       64   1.4e+01     46.2%      1.01
       2217         1465    +5.482570e+03    +2.950953e+03       48   1.4e+01     46.2%      1.04
D      2309         1507    +5.475400e+03    +2.951120e+03       98   1.4e+01     46.1%      1.07
D      2376         1538    +5.473940e+03    +2.951440e+03       65   1.4e+01     46.1%      1.10
D      2426         1570    +5.472890e+03    +2.951440e+03       64   1.4e+01     46.1%      1.12
D      2426         1570    +5.467850e+03    +2.951440e+03       65   1.4e+01     46.0%      1.13
D      2448         1592    +5.466440e+03    +2.951440e+03       99   1.4e+01     46.0%      1.14
D      2598         1668    +5.462520e+03    +2.952777e+03      100   1.4e+01     45.9%      1.21
D      2710         1754    +5.457480e+03    +2.953089e+03      101   1.4e+01     45.9%      1.27
       3906         2482    +5.457480e+03    +2.960186e+03       58   1.4e+01     45.8%      2.04
D      4525         2815    +5.451640e+03    +2.962196e+03       98   1.4e+01     45.7%      2.48
D      4556         2834    +5.449060e+03    +2.962296e+03       65   1.4e+01     45.6%      2.52
D      4614         2874    +5.441560e+03    +2.962548e+03       99   1.4e+01     45.6%      2.56
D      4669         2911    +5.440100e+03    +2.962661e+03       66   1.4e+01     45.5%      2.60
D      4703         2945    +5.432600e+03    +2.962661e+03      100   1.4e+01     45.5%      2.65
D      4791         3007    +5.428680e+03    +2.962687e+03      101   1.4e+01     45.4%      2.71
D      4879         3055    +5.423640e+03    +2.963039e+03      102   1.4e+01     45.4%      2.79
       5125         3209    +5.423640e+03    +2.963375e+03       55   1.4e+01     45.4%      3.04
       6405         3923    +5.423640e+03    +2.964850e+03       74   1.4e+01     45.3%      4.05
D      6556         4018    +5.420310e+03    +2.965093e+03      105   1.4e+01     45.3%      4.17
D      6625         4055    +5.415270e+03    +2.965457e+03      106   1.4e+01     45.2%      4.25
       7456         4572    +5.415270e+03    +2.966392e+03       80   1.4e+01     45.2%      5.05
D      8471         5255    +5.411070e+03    +2.967001e+03      109   1.4e+01     45.2%      6.02
       8517         5283    +5.411070e+03    +2.967002e+03       81   1.4e+01     45.2%      6.05
D      8652         5370    +5.407150e+03    +2.967078e+03      110   1.4e+01     45.1%      6.13
D      8752         5444    +5.402110e+03    +2.967185e+03      111   1.4e+01     45.1%      6.22
       9795         6097    +5.402110e+03    +2.967811e+03       80   1.4e+01     45.1%      7.05
H                           +5.250480e+03    +2.968327e+03                        43.5%      7.41
      11954         7431    +5.250480e+03    +2.969203e+03       57   1.4e+01     43.4%      8.05
H                           +5.244360e+03    +2.969205e+03                        43.4%      8.07
H                           +5.209540e+03    +2.969481e+03                        43.0%      8.52
H                           +5.155110e+03    +2.969735e+03                        42.4%      8.94
H                           +4.917080e+03    +2.969780e+03                        39.6%      8.98
      13932         8712    +4.917080e+03    +2.969830e+03       57   1.4e+01     39.6%      9.05
H                           +4.884120e+03    +2.969947e+03                        39.2%      9.22
H                           +4.831740e+03    +2.970001e+03                        38.5%      9.31
      15967         9837    +4.831740e+03    +2.970457e+03       71   1.4e+01     38.5%     10.06
H                           +4.592950e+03    +2.970963e+03                        35.3%     10.62
      18152        11040    +4.592950e+03    +2.971198e+03       46   1.4e+01     35.3%     11.06
H                           +4.498700e+03    +2.971221e+03                        34.0%     11.12
H                           +4.185130e+03    +2.971855e+03                        29.0%     12.01
      21441        12657    +4.185130e+03    +2.971882e+03       57   1.3e+01     29.0%     12.07
      24616        14172    +4.185130e+03    +2.972736e+03       49   1.3e+01     29.0%     13.07
      27152        15356    +4.185130e+03    +2.973213e+03       56   1.3e+01     29.0%     14.07
      29607        16537    +4.185130e+03    +2.973588e+03       67   1.3e+01     28.9%     15.07
      31945        17679    +4.185130e+03    +2.973986e+03       55   1.3e+01     28.9%     16.07
      34644        18936    +4.185130e+03    +2.974342e+03       77   1.2e+01     28.9%     17.07
      37205        20155    +4.185130e+03    +2.974594e+03       73   1.2e+01     28.9%     18.07
      39922        21356    +4.185130e+03    +2.974848e+03       65   1.2e+01     28.9%     19.07
      42042        22328    +4.185130e+03    +2.975044e+03       67   1.2e+01     28.9%     20.07
      43571        23003    +4.185130e+03    +2.975196e+03       54   1.2e+01     28.9%     21.08
      45626        23980    +4.185130e+03    +2.975387e+03       55   1.2e+01     28.9%     22.08
      48202        25172    +4.185130e+03    +2.975639e+03       54   1.2e+01     28.9%     23.08
      50865        26429    +4.185130e+03    +2.975816e+03       38   1.2e+01     28.9%     24.09
      53558        27628    +4.185130e+03    +2.976024e+03       64   1.2e+01     28.9%     25.09
      56233        28851    +4.185130e+03    +2.976263e+03       59   1.2e+01     28.9%     26.10
      58678        30060    +4.185130e+03    +2.976485e+03       63   1.2e+01     28.9%     27.10
      61337        31329    +4.185130e+03    +2.976675e+03       60   1.2e+01     28.9%     28.10
      63996        32518    +4.185130e+03    +2.976819e+03       59   1.2e+01     28.9%     29.10
      66368        33634    +4.185130e+03    +2.977045e+03       54   1.2e+01     28.9%     30.11
      68067        34377    +4.185130e+03    +2.977162e+03       70   1.2e+01     28.9%     31.11
      69773        35119    +4.185130e+03    +2.977289e+03       53   1.2e+01     28.9%     32.11
      71521        35913    +4.185130e+03    +2.977423e+03       51   1.2e+01     28.9%     33.12
      73282        36758    +4.185130e+03    +2.977548e+03       61   1.1e+01     28.9%     34.12
      75881        37881    +4.185130e+03    +2.977692e+03       62   1.1e+01     28.9%     35.12
      77291        38563    +4.185130e+03    +2.977774e+03       67   1.1e+01     28.8%     36.12
      78674        39196    +4.185130e+03    +2.977841e+03       67   1.1e+01     28.8%     37.12
      80314        39918    +4.185130e+03    +2.977956e+03       50   1.1e+01     28.8%     38.18
      82578        40952    +4.185130e+03    +2.978053e+03       66   1.1e+01     28.8%     39.18
      84859        42001    +4.185130e+03    +2.978164e+03       68   1.1e+01     28.8%     40.18
      87524        43200    +4.185130e+03    +2.978298e+03       60   1.1e+01     28.8%     41.19
      90029        44327    +4.185130e+03    +2.978403e+03       68   1.1e+01     28.8%     42.19
      92791        45667    +4.185130e+03    +2.978569e+03       58   1.1e+01     28.8%     43.19
      96314        47244    +4.185130e+03    +2.978732e+03       53   1.1e+01     28.8%     44.19
      99265        48583    +4.185130e+03    +2.978871e+03       90   1.1e+01     28.8%     45.19
     101838        49776    +4.185130e+03    +2.979004e+03       64   1.1e+01     28.8%     46.19
     104175        50881    +4.185130e+03    +2.979109e+03       53   1.1e+01     28.8%     47.20
     106389        51967    +4.185130e+03    +2.979189e+03       47   1.1e+01     28.8%     48.20
     108869        53019    +4.185130e+03    +2.979291e+03       61   1.1e+01     28.8%     49.20
     111648        54286    +4.185130e+03    +2.979391e+03       75   1.1e+01     28.8%     50.20
     113788        55360    +4.185130e+03    +2.979481e+03       68   1.1e+01     28.8%     51.21
     115356        56082    +4.185130e+03    +2.979540e+03       39   1.1e+01     28.8%     52.22
     117041        56887    +4.185130e+03    +2.979602e+03       58   1.1e+01     28.8%     53.22
     118709        57647    +4.185130e+03    +2.979661e+03       71   1.1e+01     28.8%     54.22
     120496        58412    +4.185130e+03    +2.979715e+03       60   1.1e+01     28.8%     55.22
     122262        59194    +4.185130e+03    +2.979767e+03       48   1.1e+01     28.8%     56.23
     124111        59999    +4.185130e+03    +2.979833e+03       74   1.1e+01     28.8%     57.23
     125928        60838    +4.185130e+03    +2.979904e+03       59   1.1e+01     28.8%     58.23
     127606        61600    +4.185130e+03    +2.979961e+03       74   1.1e+01     28.8%     59.24
Time limit reached. Stopping the solver...
Explored 128504 nodes in 59.85s.
Absolute Gap 1.205137e+03 Objective 4.1851299990713587e+03 Lower Bound 2.9799933322080287e+03
The feasibility does not match on scaled and unscaled problems. To overcome this issue, please provide a more numerically stable problem.
Post-solve status: Post solved solution violates constraints. This is most likely due to different tolerances.
The solution is not feasible after post solve
Solution objective: 4185.129999 , relative_mip_gap 0.287957 solution_bound 2979.993332 presolve_time 0.077081 total_solve_time 60.010936 max constraint violation 0.000037 max int violation 0.000000 max var bounds violation 0.000000 nodes 128504 simplex_iterations 1432456
cuOpt version: 25.12.0, git hash: , host arch: x86_64, device archs: 75-real,80-real,86-real,90a-real,100f-real,120a-real,120
CPU: AMD EPYC 7313P 16-Core Processor, threads (physical/logical): 12/12, RAM: 41.56 GiB
CUDA 13.0, device: NVIDIA H100 PCIe (ID 0), VRAM: 79.18 GiB
CUDA device UUID: 7dffffff85ffffffe403-78ffffffd8-ffff

Solving a problem with 63 constraints, 63 variables (53 integers), and 2016 nonzeros
Problem scaling:
Objective coefficents range:          [1e+00, 1e+00]
Constraint matrix coefficients range: [1e+00, 1e+00]
Constraint rhs / bounds range:        [0e+00, 8e+00]
Variable bounds range:                [0e+00, 1e+00]

Original problem: 63 constraints, 63 variables, 2016 nonzeros
Calling Papilo presolver
Presolve status: did not result in any changes
Presolve removed: 0 constraints, 0 variables, 0 nonzeros
Presolved problem: 63 constraints, 63 variables, 2016 nonzeros
Papilo presolve time: 0.004981
Objective offset 0.000000 scaling_factor 1.000000
Running presolve!
After trivial presolve: 63 constraints, 63 variables, objective offset 0.000000.
Using 7 CPU threads for B&B
Solving LP root relaxation
Scaling matrix. Maximum column norm 1.000000e+00, minimum column norm 1.767767e-01
Dual Simplex Phase 1
Dual feasible solution found.
Dual Simplex Phase 2
 Iter     Objective           Num Inf.  Sum Inf.     Perturb  Time
    1 +8.0000000000000000e+00      32 8.39687500e+01 0.00e+00 0.00
Removed perturbation of 2.44e-06.


Root relaxation solution found in 70 iterations and 0.00s
Root relaxation objective +1.30000000e+01


Strong branching using 7 threads and 35 fractional variables
New solution from primal heuristics. Objective +2.600000e+01. Time 0.02
New solution from primal heuristics. Objective +2.500000e+01. Time 0.02
New solution from primal heuristics. Objective +2.400000e+01. Time 0.02
Exploring the B&B tree using 7 threads (best-first = 1, diving = 6)
 | Explored | Unexplored |    Objective    |     Bound     | Depth | Iter/Node |   Gap    |  Time  |
H                           +2.003902e+01    +1.315625e+01                        34.3%      0.07
H                           +1.925000e+01    +1.315625e+01                        31.7%      0.07
H                           +1.825000e+01    +1.315625e+01                        27.9%      0.07
H                           +1.725000e+01    +1.315625e+01                        23.7%      0.07
D        50           52    +1.700000e+01    +1.315625e+01       26   3.6e+01     22.6%      0.08
D        54           56    +1.650000e+01    +1.315625e+01       31   3.4e+01     20.3%      0.08
D        56           58    +1.600000e+01    +1.315625e+01       29   3.4e+01     17.8%      0.08
D        59           59    +1.600000e+01    +1.317857e+01       27   3.3e+01     17.6%      0.09
D        60           60    +1.600000e+01    +1.317857e+01       26   3.4e+01     17.6%      0.09
D        61           61    +1.600000e+01    +1.317857e+01       35   3.5e+01     17.6%      0.09
D        62           62    +1.550000e+01    +1.317857e+01       27   3.5e+01     15.0%      0.10
D       177          169    +1.550000e+01    +1.325000e+01       31   3.3e+01     14.5%      0.25
D       813          709    +1.500000e+01    +1.375000e+01       29   2.3e+01      8.3%      0.81
       1028          890    +1.500000e+01    +1.375000e+01       10   2.4e+01      8.3%      1.03
       2028         1760    +1.500000e+01    +1.375000e+01       29   2.3e+01      8.3%      2.14
D      2866         2468    +1.500000e+01    +1.375000e+01       28   2.4e+01      8.3%      3.20
       3028         2604    +1.500000e+01    +1.375000e+01       23   2.4e+01      8.3%      3.34
       4056         3484    +1.500000e+01    +1.375000e+01       13   2.4e+01      8.3%      4.34
       5092         4348    +1.500000e+01    +1.375000e+01       11   2.3e+01      8.3%      5.34
       6092         5198    +1.500000e+01    +1.375000e+01       27   2.3e+01      8.3%      6.60
timeout: the monitored command dumped core
ci/test_cpp.sh: line 49:  1555 Segmentation fault      timeout 20m "${gt}" --gtest_output=xml:"{RAPIDS_TESTS_DIR}"
Running gtest MPS_PARSER_TEST
Running main() from gmock_main.cc

Metadata

Metadata

Labels

awaiting responseThis expects a response from maintainer or contributor depending on who requested in last comment.bugSomething isn't working

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions