Add warning when using projections and get model increase in tr subproblem

Author: mjhough

dfols/controller.py

@@ -43,6 +43,7 @@
            'EXIT_INPUT_ERROR', 'EXIT_TR_INCREASE_ERROR', 'EXIT_LINALG_ERROR', 'EXIT_FALSE_SUCCESS_WARNING',
            'EXIT_AUTO_DETECT_RESTART_WARNING']
 
+EXIT_TR_INCREASE_WARNING = 5  # warning, TR increase in proj constrained case - likely due to multiple active constraints
 EXIT_AUTO_DETECT_RESTART_WARNING = 4  # warning, auto-detected restart criteria
 EXIT_FALSE_SUCCESS_WARNING = 3  # warning, maximum fake successful steps reached
 EXIT_SLOW_WARNING = 2  # warning, maximum number of slow (successful) iterations reached
@@ -70,6 +71,8 @@ def message(self, with_stem=True):
             return "Warning (slow progress): " + self.msg
         elif self.flag == EXIT_MAXFUN_WARNING:
             return "Warning (max evals): " + self.msg
+        elif self.flag == EXIT_TR_INCREASE_WARNING:
+            return "Warning (trust region increase): " + self.msg
         elif self.flag == EXIT_INPUT_ERROR:
             return "Error (bad input): " + self.msg
         elif self.flag == EXIT_TR_INCREASE_ERROR:
@@ -82,7 +85,7 @@ def message(self, with_stem=True):
             return "Unknown exit flag: " + self.msg
 
     def able_to_do_restart(self):
-        if self.flag in [EXIT_TR_INCREASE_ERROR, EXIT_LINALG_ERROR, EXIT_SLOW_WARNING, EXIT_AUTO_DETECT_RESTART_WARNING]:
+        if self.flag in [EXIT_TR_INCREASE_ERROR, EXIT_TR_INCREASE_WARNING, EXIT_LINALG_ERROR, EXIT_SLOW_WARNING, EXIT_AUTO_DETECT_RESTART_WARNING]:
             return True
         elif self.flag in [EXIT_MAXFUN_WARNING, EXIT_INPUT_ERROR]:
             return False
@@ -616,7 +619,10 @@ def calculate_ratio(self, current_iter, rvec_list, d, gopt, H):
         if min(sqrt(sumsq(d)), self.delta) > self.rho:  # if ||d|| >= rho, successful!
             self.last_successful_iter = current_iter
         if pred_reduction < 0.0:
-            exit_info = ExitInformation(EXIT_TR_INCREASE_ERROR, "Trust region step gave model increase")
+            if len(self.model.projections) > 1: # if we are using multiple projections, only warn since likely due to constraint intersection
+                exit_info = ExitInformation(EXIT_TR_INCREASE_WARNING, "Either multiple constraints are active or trust region step gave model increase")
+            else:
+                exit_info = ExitInformation(EXIT_TR_INCREASE_ERROR, "Either rust region step gave model increase")
 
         ratio = actual_reduction / pred_reduction
         return ratio, exit_info

docs/build/doctrees/environment.pickle

@@ -48,6 +48,7 @@ The possible values of :code:`soln.flag` are defined by the following variables:
 * :code:`soln.EXIT_MAXFUN_WARNING` - maximum allowed objective evaluations reached. This is the most likely return value when using multiple restarts.
 * :code:`soln.EXIT_SLOW_WARNING` - maximum number of slow iterations reached.
 * :code:`soln.EXIT_FALSE_SUCCESS_WARNING` - DFO-LS reached the maximum number of restarts which decreased the objective, but to a worse value than was found in a previous run.
+* :code:`soln.EXIT_TR_INCREASE_WARNING` - model increase when solving the trust region subproblem with multiple arbitrary constraints.
 * :code:`soln.EXIT_INPUT_ERROR` - error in the inputs.
 * :code:`soln.EXIT_TR_INCREASE_ERROR` - error occurred when solving the trust region subproblem.
 * :code:`soln.EXIT_LINALG_ERROR` - linear algebra error, e.g. the interpolation points produced a singular linear system.
@@ -180,7 +181,7 @@ However, we also get a warning that our starting point was outside of the bounds
 
 DFO-LS automatically fixes this, and moves :math:`x_0` to a point within the bounds, in this case :math:`x_0=(-1.2,0.85)`.
 
-If we want more complex constraints, we can instead write
+If we want more complex constraints, we can instead write something like the following:
 
   .. code-block:: python
   
@@ -198,19 +199,20 @@ If we want more complex constraints, we can instead write
       # Call DFO-LS (with box and ball constraints)
       soln = dfols.solve(rosenbrock, x0, projections=[pball,pbox])
 
-DFO-LS correctly finds the solution to this constrained problem too:
+DFO-LS correctly finds the solution to this constrained problem too. Note that we get a warning because the step computed in the trust region subproblem
+gave an increase in the model. This is common in the case where multiple constraints are active at the optimal point.
 
   .. code-block:: none
-  
+
       ****** DFO-LS Results ******
-      Solution xmin = [0.9        1.15358984]
-      Residual vector = [3.43589838 0.1       ]
-      Objective value f(xmin) = 11.81539771
+      Solution xmin = [0.9        1.15359245]
+      Residual vector = [3.43592448 0.1       ]
+      Objective value f(xmin) = 11.81557703
       Needed 10 objective evaluations (at 10 points)
       Approximate Jacobian = [[-1.79826221e+01  1.00004412e+01]
-       [-1.00000000e+00  6.81262102e-15]]
-      Exit flag = 0
-      Success: rho has reached rhoend
+       [-1.00000000e+00 -1.81976605e-15]]
+      Exit flag = 5
+      Warning (trust region increase): Either multiple constraints are active or trust region step gave model increase
       ****************************
 
 

docs/build/doctrees/userguide.doctree

@@ -211,6 +211,7 @@ <h2>How to use DFO-LS<a class="headerlink" href="#how-to-use-dfo-ls" title="Perm
 <li><p><code class="code docutils literal notranslate"><span class="pre">soln.EXIT_MAXFUN_WARNING</span></code> - maximum allowed objective evaluations reached. This is the most likely return value when using multiple restarts.</p></li>
 <li><p><code class="code docutils literal notranslate"><span class="pre">soln.EXIT_SLOW_WARNING</span></code> - maximum number of slow iterations reached.</p></li>
 <li><p><code class="code docutils literal notranslate"><span class="pre">soln.EXIT_FALSE_SUCCESS_WARNING</span></code> - DFO-LS reached the maximum number of restarts which decreased the objective, but to a worse value than was found in a previous run.</p></li>
+<li><p><code class="code docutils literal notranslate"><span class="pre">soln.EXIT_TR_INCREASE_WARNING</span></code> - model increase when solving the trust region subproblem with multiple arbitrary constraints.</p></li>
 <li><p><code class="code docutils literal notranslate"><span class="pre">soln.EXIT_INPUT_ERROR</span></code> - error in the inputs.</p></li>
 <li><p><code class="code docutils literal notranslate"><span class="pre">soln.EXIT_TR_INCREASE_ERROR</span></code> - error occurred when solving the trust region subproblem.</p></li>
 <li><p><code class="code docutils literal notranslate"><span class="pre">soln.EXIT_LINALG_ERROR</span></code> - linear algebra error, e.g. the interpolation points produced a singular linear system.</p></li>
@@ -336,7 +337,7 @@ <h2>Adding Constraints and More Output<a class="headerlink" href="#adding-constr
 </div>
 </div></blockquote>
 <p>DFO-LS automatically fixes this, and moves <span class="math notranslate nohighlight">\(x_0\)</span> to a point within the bounds, in this case <span class="math notranslate nohighlight">\(x_0=(-1.2,0.85)\)</span>.</p>
-<p>If we want more complex constraints, we can instead write</p>
+<p>If we want more complex constraints, we can instead write something like the following:</p>
 <blockquote>
 <div><div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Define the projection functions</span>
 <span class="k">def</span> <span class="nf">pball</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
@@ -354,17 +355,18 @@ <h2>Adding Constraints and More Output<a class="headerlink" href="#adding-constr
 </pre></div>
 </div>
 </div></blockquote>
-<p>DFO-LS correctly finds the solution to this constrained problem too:</p>
+<p>DFO-LS correctly finds the solution to this constrained problem too. Note that we get a warning because the step computed in the trust region subproblem
+gave an increase in the model. This is common in the case where multiple constraints are active at the optimal point.</p>
 <blockquote>
 <div><div class="highlight-none notranslate"><div class="highlight"><pre><span></span>****** DFO-LS Results ******
-Solution xmin = [0.9        1.15358984]
-Residual vector = [3.43589838 0.1       ]
-Objective value f(xmin) = 11.81539771
+Solution xmin = [0.9        1.15359245]
+Residual vector = [3.43592448 0.1       ]
+Objective value f(xmin) = 11.81557703
 Needed 10 objective evaluations (at 10 points)
 Approximate Jacobian = [[-1.79826221e+01  1.00004412e+01]
- [-1.00000000e+00  6.81262102e-15]]
-Exit flag = 0
-Success: rho has reached rhoend
+ [-1.00000000e+00 -1.81976605e-15]]
+Exit flag = 5
+Warning (trust region increase): Either multiple constraints are active or trust region step gave model increase
 ****************************
 </pre></div>
 </div>

docs/build/html/_sources/userguide.rst.txt

@@ -46,12 +46,12 @@
 \newlabel{info:parameter-fitting}{{2.2}{5}{Parameter Fitting}{section.2.2}{}}
 \@writefile{toc}{\contentsline {section}{\numberline {2.3}Solving Nonlinear Systems of Equations}{6}{section.2.3}\protected@file@percent }
 \newlabel{info:solving-nonlinear-systems-of-equations}{{2.3}{6}{Solving Nonlinear Systems of Equations}{section.2.3}{}}
-\citation{info:cfmr2018}
+\citation{userguide:cfmr2018}
 \@writefile{toc}{\contentsline {section}{\numberline {2.4}Details of the DFO\sphinxhyphen {}LS Algorithm}{7}{section.2.4}\protected@file@percent }
 \newlabel{info:details-of-the-dfo-ls-algorithm}{{2.4}{7}{Details of the DFO\sphinxhyphen {}LS Algorithm}{section.2.4}{}}
 \@writefile{toc}{\contentsline {section}{\numberline {2.5}References}{7}{section.2.5}\protected@file@percent }
 \newlabel{info:references}{{2.5}{7}{References}{section.2.5}{}}
-\citation{info:cfmr2018}
+\citation{userguide:cfmr2018}
 \@writefile{toc}{\contentsline {chapter}{\numberline {3}Using DFO\sphinxhyphen {}LS}{9}{chapter.3}\protected@file@percent }
 \@writefile{lof}{\addvspace {10\p@ }}
 \@writefile{lot}{\addvspace {10\p@ }}
@@ -75,7 +75,7 @@
 \newlabel{userguide:example-solving-a-nonlinear-system-of-equations}{{3.8}{18}{Example: Solving a Nonlinear System of Equations}{section.3.8}{}}
 \@writefile{toc}{\contentsline {section}{\numberline {3.9}References}{19}{section.3.9}\protected@file@percent }
 \newlabel{userguide:references}{{3.9}{19}{References}{section.3.9}{}}
-\citation{info:cfmr2018}
+\citation{userguide:cfmr2018}
 \@writefile{toc}{\contentsline {chapter}{\numberline {4}Advanced Usage}{21}{chapter.4}\protected@file@percent }
 \@writefile{lof}{\addvspace {10\p@ }}
 \@writefile{lot}{\addvspace {10\p@ }}

docs/build/html/objects.inv

@@ -1,10 +1,10 @@
 # Fdb version 3
-["makeindex DFOLS.idx"] 1533896586 "DFOLS.idx" "DFOLS.ind" "DFOLS" 1634440875
-  "DFOLS.idx" 1634440874 0 d41d8cd98f00b204e9800998ecf8427e "pdflatex"
+["makeindex DFOLS.idx"] 1533896586 "DFOLS.idx" "DFOLS.ind" "DFOLS" 1635178686
+  "DFOLS.idx" 1635178685 0 d41d8cd98f00b204e9800998ecf8427e "pdflatex"
   (generated)
-  "DFOLS.ind"
   "DFOLS.ilg"
-["pdflatex"] 1634440872 "DFOLS.tex" "DFOLS.pdf" "DFOLS" 1634440875
+  "DFOLS.ind"
+["pdflatex"] 1635178684 "DFOLS.tex" "DFOLS.pdf" "DFOLS" 1635178686
   "/etc/texmf/web2c/texmf.cnf" 1624242784 475 c0e671620eb5563b2130f56340a5fde8 ""
   "/usr/share/texlive/texmf-dist/fonts/enc/dvips/base/8r.enc" 1165713224 4850 80dc9bab7f31fb78a000ccfed0e27cab ""
   "/usr/share/texlive/texmf-dist/fonts/map/fontname/texfonts.map" 1577235249 3524 cb3e574dea2d1052e39280babc910dc8 ""
@@ -173,22 +173,22 @@
   "/usr/share/texmf/web2c/texmf.cnf" 1613593815 38841 799d1dd9682a55ce442e10c99777ecc1 ""
   "/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map" 1624243066 5160710 ecf427ae8fa19139d8691f526e47bb9b ""
   "/var/lib/texmf/web2c/pdftex/pdflatex.fmt" 1624243094 2569235 be2903e90bfe54d2770ce0f508f15937 ""
-  "DFOLS.aux" 1634440874 14077 a57e9cbc321455cf71aa897afa283d2c "pdflatex"
+  "DFOLS.aux" 1635178686 14092 9d9d9e97af3301fe1a6671e122c87eb2 "pdflatex"
   "DFOLS.ind" 1610358580 0 d41d8cd98f00b204e9800998ecf8427e "makeindex DFOLS.idx"
-  "DFOLS.out" 1634440874 9566 4c15cb7f3428ba1015be450cf20ae095 "pdflatex"
-  "DFOLS.tex" 1634440863 88436 83d524cb5461f8777e2e649ff2ddefdb ""
-  "DFOLS.toc" 1634440874 4715 c361968e0fbff7dd069132047d694ae5 "pdflatex"
+  "DFOLS.out" 1635178686 9566 4c15cb7f3428ba1015be450cf20ae095 "pdflatex"
+  "DFOLS.tex" 1635178681 88954 c0e52f684f29b5b7b470a229b4c94cfb ""
+  "DFOLS.toc" 1635178686 4715 c361968e0fbff7dd069132047d694ae5 "pdflatex"
   "data_fitting.png" 1610358580 29893 211bb1c28ea25d47c8c0990fbf39c55c ""
   "footnotehyper-sphinx.sty" 1610358580 8888 1bbd7bdeae8c8bed1d10d551bddb1cc9 ""
   "sphinx.sty" 1633985769 82020 a38700b8aa22dfd94a8a5b905e69be73 ""
-  "sphinxhighlight.sty" 1634440869 8137 38a433148fcb7611515a989ff1750dd5 ""
+  "sphinxhighlight.sty" 1635178681 8137 38a433148fcb7611515a989ff1750dd5 ""
   "sphinxmanual.cls" 1610358580 4236 124cd90deb92742b5d3922bfc2cd70c0 ""
-  "sphinxmessages.sty" 1634440869 745 3f5fcd6cdd7964ed608767954a8ced6f ""
+  "sphinxmessages.sty" 1635178681 745 3f5fcd6cdd7964ed608767954a8ced6f ""
   "sphinxmulticell.sty" 1610358580 14606 0b6edc2b1a83546ed92026d1f6a311b5 ""
   (generated)
-  "DFOLS.aux"
-  "DFOLS.log"
   "DFOLS.idx"
+  "DFOLS.aux"
   "DFOLS.toc"
   "DFOLS.pdf"
+  "DFOLS.log"
   "DFOLS.out"

docs/build/html/searchindex.js

@@ -1,4 +1,4 @@
-This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020/Debian) (preloaded format=pdflatex 2021.6.21)  16 OCT 2021 23:21
+This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020/Debian) (preloaded format=pdflatex 2021.6.21)  25 OCT 2021 12:18
 entering extended mode
  restricted \write18 enabled.
  %&-line parsing enabled.
@@ -698,52 +698,48 @@ File: ts1pcr.fd 2001/06/04 font definitions for TS1/pcr.
 
 Package textcomp Warning: Symbol \textasciigrave not provided by
 (textcomp)                font family pcr in TS1 encoding.
-(textcomp)                Default family used instead on input line 756.
+(textcomp)                Default family used instead on input line 760.
 
 LaTeX Font Info:    Trying to load font information for T1+cmtt on input line 7
-56.
+60.
 (/usr/share/texlive/texmf-dist/tex/latex/base/t1cmtt.fd
 File: t1cmtt.fd 2019/12/16 v2.5j Standard LaTeX font definitions
 )
 LaTeX Font Info:    Trying to load font information for TS1+cmtt on input line 
-756.
+760.
 
 (/usr/share/texlive/texmf-dist/tex/latex/base/ts1cmtt.fd
 File: ts1cmtt.fd 2019/12/16 v2.5j Standard LaTeX font definitions
 )
 
 Package textcomp Warning: Symbol \textasciigrave not provided by
 (textcomp)                font family pcr in TS1 encoding.
-(textcomp)                Default family used instead on input line 756.
-
-
-Overfull \vbox (0.68164pt too high) detected at line 784
- []
+(textcomp)                Default family used instead on input line 760.
 
 [15] [16]
 <data_fitting.png, id=424, 524.6802pt x 403.2666pt>
 File: data_fitting.png Graphic file (type png)
 <use data_fitting.png>
-Package pdftex.def Info: data_fitting.png  used on input line 857.
+Package pdftex.def Info: data_fitting.png  used on input line 861.
 (pdftex.def)             Requested size: 352.31625pt x 270.79639pt.
  [17 <./data_fitting.png>] [18] [19] [20
 
 ]
 Chapter 4.
 [21] [22]
-Underfull \hbox (badness 10000) in paragraph at lines 1085--1086
+Underfull \hbox (badness 10000) in paragraph at lines 1089--1090
 []\T1/pcr/m/n/10 regression.increase_num_extra_steps_with_restart \T1/ptm/m/n/1
 0 - The amount to in-crease
  []
 
 
-Underfull \hbox (badness 10000) in paragraph at lines 1120--1121
+Underfull \hbox (badness 10000) in paragraph at lines 1124--1125
 []\T1/pcr/m/n/10 restarts.hard.increase_ndirs_initial_amt \T1/ptm/m/n/10 - Amou
 nt to in-crease \T1/pcr/m/n/10 growing.
  []
 
 [23]
-Underfull \hbox (badness 8000) in paragraph at lines 1155--1156
+Underfull \hbox (badness 8000) in paragraph at lines 1159--1160
 \T1/ptm/m/n/10 De-fault is \T1/pcr/m/n/10 False \T1/ptm/m/n/10 if $\OML/cmm/m/i
 t/10 m \OMS/cmsy/m/n/10 ^^U \OML/cmm/m/it/10 n$ \T1/ptm/m/n/10 and \T1/pcr/m/n/
 10 True \T1/ptm/m/n/10 oth-er-wise (op-po-site to \T1/pcr/m/n/10 growing.full_r
@@ -773,7 +769,7 @@ Package rerunfilecheck Info: File `DFOLS.out' has not changed.
 Here is how much of TeX's memory you used:
  13946 strings out of 479304
  199409 string characters out of 5869779
- 597748 words of memory out of 5000000
+ 604138 words of memory out of 5000000
  30696 multiletter control sequences out of 15000+600000
  452760 words of font info for 89 fonts, out of 8000000 for 9000
  1142 hyphenation exceptions out of 8191
@@ -799,7 +795,7 @@ exmf-dist/fonts/type1/urw/helvetic/uhvbo8a.pfb></usr/share/texlive/texmf-dist/f
 onts/type1/urw/times/utmb8a.pfb></usr/share/texlive/texmf-dist/fonts/type1/urw/
 times/utmr8a.pfb></usr/share/texlive/texmf-dist/fonts/type1/urw/times/utmri8a.p
 fb>
-Output written on DFOLS.pdf (37 pages, 321015 bytes).
+Output written on DFOLS.pdf (37 pages, 321478 bytes).
 PDF statistics:
  616 PDF objects out of 1000 (max. 8388607)
  543 compressed objects within 6 object streams