Generalized Cauchy direction and box constraints

[mateuszbaran]

This is a WIP PR that implements the GCP idea to L-BFGS and maybe some other solvers, so that they can be used in the presence of box constraints on the Euclidean part of the manifold. The constraints are handled similarly to L-BFGS-B (although not all Euclidean tricks are applied).

TODO:

• a full example of working with a box domain
• code coverage

The implementation was prepared in collaboration with @paprzybysz .

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (88c7db6) to head (a192dcc).

Additional details and impacted files

@@            Coverage Diff             @@
##            master      #554    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           91        92     +1     
  Lines         9975     10661   +686     
==========================================
+ Hits          9975     10661   +686     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mateuszbaran]

I've generalized the GCD subsolver to handle Hyperrectangle at any position, and also potentially multiple hyperrectangles in a product manifold. This comes at a slight performance cost because now we have dynamic dispatch when accessing a component that is avoidable but only with a much more convoluted code. I'm not exactly certain this is the right call but it makes the solver more generic and harder to use wrongly.

[kellertuer]

Thanks for fixing a few typos along the way and tightening a few parameter types along the way (I am usually a bit unsure when to type kwargs and when not.

Interesting that the *get_cost(problem, state)` did not yet exist, but sure I see points where that might be nice to have.

Besides that I read the new HZ part (but not more yet) and added a few comments.

[mateuszbaran]

I think I've roughly addressed all recent.

[mateuszbaran]

I have also added a simple direction-based stepsize limiting that can be used in solvers that don't approximate the Hessian to limit the stepsize.