Upgrade ensmallen to 2 10 3 (#16)

coatless · web-flow · commit 1fa380a4c186 · 2019-09-30T16:53:00.000-05:00
* Update ensmallen to 2.10.2

* Update news entry

* Bump version

* Upgrade to ensmallen 2.10.3

* Bump version and add vignette setup

* Update news with new entries.

* Block doc

* Add a vignette

* Add a ChangeLog entry

* Suppress building vignettes on TravisCI

* Remove empty space

* Better prose

* Add a vignette trick

* Remove -I

* Add ChangeLog
diff --git a/.gitignore b/.gitignore
@@ -6,3 +6,4 @@
 src/*.o
 src/*.so
 src/*.dll
+inst/doc
diff --git a/.travis.yml b/.travis.yml
@@ -13,6 +13,8 @@ cache: packages
 env:
   global:
   - MAKEFLAGS="-j 2"
+  - R_BUILD_ARGS="--no-build-vignettes"
+  - R_CHECK_ARGS="--no-build-vignettes"
   - _R_CHECK_FORCE_SUGGESTS=0
 
 # Enable a build matrix that individually tests the package against the
diff --git a/ChangeLog b/ChangeLog
@@ -1,3 +1,20 @@
+2019-09-30  James Balamuta  <balamut2@illinois.edu>
+
+	* vignettes/using-rcppensmallen.Rmd: Improved wording.
+
+	* .travis.yml: Disabled vignette building on TravisCI
+
+2019-09-27  James Balamuta  <balamut2@illinois.edu>
+
+	* DESCRIPTION (Version, Date): Release 2.10.3
+
+	* NEWS.md: Update for Ensmallen release 2.10.3
+
+	* inst/include/ensmallen_bits: Upgraded to Ensmallen 2.10.3
+	* inst/include/ensmallen.hpp: ditto
+
+	* vignettes/using-rcppensmallen.Rmd: Added a vignette describing use.
+
 2019-09-07  James Balamuta  <balamut2@illinois.edu>
 
 	* DESCRIPTION (Version, Date): Release 2.10.0 and armadillo dependency
diff --git a/DESCRIPTION b/DESCRIPTION
@@ -1,6 +1,6 @@
 Package: RcppEnsmallen
 Title: Header-Only C++ Mathematical Optimization Library for 'Armadillo'
-Version: 0.2.10.0.2
+Version: 0.2.10.3.1
 Authors@R: c(
     person("James Joseph", "Balamuta", email = "balamut2@illinois.edu", 
            role = c("aut", "cre", "cph"), 
@@ -34,3 +34,7 @@ Imports: Rcpp
 RoxygenNote: 6.1.1
 Roxygen: list(markdown = TRUE)
 SystemRequirements: C++11
+Suggests: 
+    knitr,
+    rmarkdown
+VignetteBuilder: knitr
diff --git a/NEWS.md b/NEWS.md
@@ -1,3 +1,21 @@
+# RcppEnsmallen 0.2.10.3.1
+
+- Upgraded to ensmallen 2.10.3: "Fried Chicken" (2019-09-26)
+  - Add release script to rel/ for maintainers
+    ([#128](https://github.com/mlpack/ensmallen/pull/128)).
+  - Fix Armadillo version check
+    ([#133](https://github.com/mlpack/ensmallen/pull/133)).
+  - Documentation fix for callbacks
+    ([#129](https://github.com/mlpack/ensmallen/pull/129).
+  - Compatibility fixes for ensmallen 1.x
+    ([#131](https://github.com/mlpack/ensmallen/pull/131)).
+  - Fix ParallelSGD runtime bug.
+    ([#135](https://github.com/mlpack/ensmallen/pull/135)).
+  - Add additional L-BFGS convergence check
+    ([#136](https://github.com/mlpack/ensmallen/pull/136)).
+- Added vignette with worked example for linear regression and for package 
+  inclusion.
+
 # RcppEnsmallen 0.2.10.0.1
 
 - Upgraded to ensmallen 2.10.0: "Fried Chicken" (2019-09-07)
diff --git a/inst/include/ensmallen.hpp b/inst/include/ensmallen.hpp
@@ -29,8 +29,8 @@
   #error "please enable C++11/C++14 mode in your compiler"
 #endif
 
-#if ((ARMA_VERSION_MAJOR < 6) || ((ARMA_VERSION_MAJOR == 6) && (ARMA_VERSION_MINOR < 500)))
-  #error "need Armadillo version 6.500 or later"
+#if ((ARMA_VERSION_MAJOR < 8) || ((ARMA_VERSION_MAJOR == 8) && (ARMA_VERSION_MINOR < 400)))
+  #error "need Armadillo version 8.400 or later"
 #endif
 
 #include <cmath>
diff --git a/inst/include/ensmallen_bits/config.hpp b/inst/include/ensmallen_bits/config.hpp
@@ -47,10 +47,12 @@
   #define ENS_PRAGMA_OMP_PARALLEL _Pragma("omp parallel")
   #define ENS_PRAGMA_OMP_ATOMIC   _Pragma("omp atomic")
   #define ENS_PRAGMA_OMP_CRITICAL _Pragma("omp critical")
+  #define ENS_PRAGMA_OMP_CRITICAL_NAMED _Pragma("omp critical(section)")
 #else
   #define ENS_PRAGMA_OMP_PARALLEL
   #define ENS_PRAGMA_OMP_ATOMIC
   #define ENS_PRAGMA_OMP_CRITICAL
+  #define ENS_PRAGMA_OMP_CRITICAL_NAMED
 #endif
 
 
diff --git a/inst/include/ensmallen_bits/ens_version.hpp b/inst/include/ensmallen_bits/ens_version.hpp
@@ -16,7 +16,7 @@
 // The minor version is two digits so regular numerical comparisons of versions
 // work right.  The first minor version of a release is always 10.
 #define ENS_VERSION_MINOR 10
-#define ENS_VERSION_PATCH 0
+#define ENS_VERSION_PATCH 3
 // If this is a release candidate, it will be reflected in the version name
 // (i.e. the version name will be "RC1", "RC2", etc.).  Otherwise the version
 // name will typically be a seemingly arbitrary set of words that does not
diff --git a/inst/include/ensmallen_bits/function/arma_traits.hpp b/inst/include/ensmallen_bits/function/arma_traits.hpp
@@ -30,7 +30,7 @@ struct MatTypeTraits<arma::Col<eT>>
 template<typename eT>
 struct MatTypeTraits<arma::Row<eT>>
 {
-  typedef arma::Row<eT> BaseMatType;
+  typedef arma::Mat<eT> BaseMatType;
 };
 
 template<typename eT>
diff --git a/inst/include/ensmallen_bits/lbfgs/lbfgs_impl.hpp b/inst/include/ensmallen_bits/lbfgs/lbfgs_impl.hpp
@@ -409,6 +409,12 @@ L_BFGS::Optimize(FunctionType& function,
 
     // Choose the scaling factor.
     double scalingFactor = ChooseScalingFactor(itNum, gradient, s, y);
+    if (scalingFactor == 0.0)
+    {
+      Info << "L-BFGS scaling factor computed as 0 (terminating successfully)."
+          << std::endl;
+      break;
+    }
 
     // Build an approximation to the Hessian and choose the search
     // direction for the current iteration.
diff --git a/inst/include/ensmallen_bits/parallel_sgd/parallel_sgd_impl.hpp b/inst/include/ensmallen_bits/parallel_sgd/parallel_sgd_impl.hpp
@@ -39,7 +39,7 @@ inline void UpdateLocation(arma::SpMat<eT>& iterate,
                            const size_t col,
                            const eT value)
 {
-  ENS_PRAGMA_OMP_CRITICAL
+  ENS_PRAGMA_OMP_CRITICAL_NAMED
   {
     iterate(row, col) -= value;
   }
diff --git a/inst/include/ensmallen_bits/sdp/lrsdp_function.hpp b/inst/include/ensmallen_bits/sdp/lrsdp_function.hpp
@@ -93,10 +93,11 @@ class LRSDPFunction
   size_t NumConstraints() const { return sdp.NumConstraints(); }
 
   //! Get the initial point of the LRSDP.
-  template<typename MatType>
-  const MatType& GetInitialPoint() const
+  template<typename MatType = arma::mat>
+  MatType GetInitialPoint() const
   {
-    return arma::conv_to<MatType>::from(initialPoint);
+    MatType result = arma::conv_to<MatType>::from(initialPoint);
+    return result;
   }
 
   //! Return the SDP object representing the problem.
diff --git a/inst/include/ensmallen_bits/sdp/sdp.hpp b/inst/include/ensmallen_bits/sdp/sdp.hpp
@@ -133,11 +133,11 @@ class SDP
   bool HasLinearlyIndependentConstraints() const;
 
   //! Get an initial point for the primal coordinates.
-  template<typename MatType>
+  template<typename MatType = arma::mat>
   MatType GetInitialPoint() const;
 
   //! Get initial points for the primal and dual coordinates.
-  template<typename MatType>
+  template<typename MatType = arma::mat>
   void GetInitialPoints(MatType& coordinates,
                         MatType& ySparse,
                         MatType& yDense,
diff --git a/vignettes/using-rcppensmallen.Rmd b/vignettes/using-rcppensmallen.Rmd
@@ -0,0 +1,209 @@
+---
+title: "Using RcppEnsmallen"
+author: James Joseph Balamuta and Dirk Eddelbuettel
+abstract: |
+  Contained within are examples related to using RcppEnsmallen in everyday 
+  work alongside of creating an _R_ package with it. 
+output: rmarkdown::html_vignette
+vignette: >
+  %\VignetteIndexEntry{Using RcppEnsmallen}
+  %\VignetteEngine{knitr::rmarkdown}
+  %\VignetteEncoding{UTF-8}
+---
+
+```{r, include = FALSE}
+knitr::opts_chunk$set(
+  collapse = TRUE,
+  comment = "#>"
+)
+```
+
+# Overview
+
+`RcppEnsmallen` package provides an embedded copy of the
+[`ensmallen`](https://www.ensmallen.org/docs.html) 
+C++ library of optimization functions. Optimizers contained within are state
+of the art and possess a high level of code quality. Each optimizer must be 
+accessed through _C++_ by implementing the appropriate objective functions and,
+then, surfaced into _R_ through
+[`RcppArmadillo`](https://cran.r-project.org/package=RcppArmadillo). 
+Alternatively, work has been done by Dirk Schumacher in 
+[`armacmp`](https://github.com/dirkschumacher/armacmp) to automatically create
+the underlying _C++_ code from _R_.  
+
+**Note:** Optimizers in `RcppEnsmallen` only work with [`armadillo`](http://arma.sourceforge.net/docs.html)
+data structures. Thus, if using [`Eigen`](http://eigen.tuxfamily.org/index.php?title=Main_Page) through 
+[`RcppEigen`](https://cran.r-project.org/package=RcppEigen), please consider the 
+[`RcppNumerical`](https://cran.r-project.org/package=RcppNumerical)
+package.
+
+## Linear Regression 
+
+Consider the **Residual Sum of Squares**, also known as **RSS**, defined as:
+
+$$RSS\left( \beta \right) = \left( { \mathbf{y} - \mathbf{X} \beta } \right)^{T} \left( \mathbf{y} - \mathbf{X} \beta \right)$$
+
+The objective function we wish to minimize would be defined as:
+
+$$f(\beta) = (y - X\beta)^2$$
+
+The gradient is defined as:
+
+$$\frac{\partial RSS}{\partial \beta} = -2 \mathbf{X}^{T} \left(\mathbf{y} - \mathbf{X} \beta \right)$$
+
+### Two-Step Implementation
+
+When using `ensmallen` to solve this problem, we must create a _C++_ class that 
+computes both the objective function value and its gradient value either
+together or separately under member functions named as:
+
+- `Evaluate()`: Value of the objective function under the parameters.
+- `Gradient()`: Convergence to the correct value under the given parameters. 
+- `EvaluateWithGradient()`: Perform both steps at the same time. (Optional)
+
+In the Linear Regression scenario, we will define each step separately to
+emphasize the calculation occurring. Generally, the one step `EvaluateWithGradient()` 
+function will be faster than the two step variant. More details on design can be
+found on [`ensmallen` documentation page for differentiable functions](https://www.ensmallen.org/docs.html#differentiable-functions).
+
+Before writing the class, `RcppEnsmallen` requires accessing the
+library in a standalone C++ file with the follow include and Rcpp Attribute
+declarations:
+
+```{Rcpp header, eval = FALSE}
+#include <RcppEnsmallen.h>
+// [[Rcpp::depends(RcppEnsmallen)]]
+```
+
+The overaching Linear regression class should be constructed as follows: 
+
+```{Rcpp classdef, eval = FALSE}
+#include <RcppEnsmallen.h>
+// [[Rcpp::depends(RcppEnsmallen)]]
+
+// Define a differentiable objective function by implementing both Evaluate()
+// and Gradient() separately.
+class LinearRegressionFunction
+{
+public:
+  // Construct the object with the given the design 
+  // matrix and responses.
+  LinearRegressionFunction(const arma::mat& X,
+                           const arma::vec& y) :
+  X(X), y(y) { }
+
+  // Return the objective function for model parameters beta.
+  double Evaluate(const arma::mat& beta)
+  {
+      return std::pow(arma::norm(y - X * beta), 2.0);
+  }
+
+  // Compute the gradient for model parameters beta
+  void Gradient(const arma::mat& beta, arma::mat& g)
+  {
+      g = -2 * X.t() * (y - X * beta);
+  }
+
+private:
+  // The design matrix.
+  const arma::mat& X;
+  // The responses to each data point.
+  const arma::vec& y;
+};
+```
+
+From there:
+
+1. Construct a _C++_ function that exports into _R_.
+2. Within the function, determine an appropriate optimizer for the problem. 
+3. Combine the optimizer with the linear regression class to compute 
+   the solution to the problem.
+
+**Note:** Make sure to have the definition of the Linear Regression class in 
+the same _C++_ file as the exported _C++_ function into _R_. 
+
+```{Rcpp linreg, eval = FALSE}
+// [[Rcpp::export]]
+arma::mat lin_reg_lbfgs(const arma::mat& X, const arma::vec& y) {
+
+  // Construct the first objective function.
+  LinearRegressionFunction lrf(X, y);
+
+  // Create the L_BFGS optimizer with default parameters.
+  // The ens::L_BFGS type can be replaced with any ensmallen optimizer that can
+  // handle differentiable functions.
+  ens::L_BFGS lbfgs;
+
+  lbfgs.MaxIterations() = 10;
+
+  // Create a starting point for our optimization randomly.
+  // The model has p parameters, so the shape is p x 1.
+  arma::mat beta(X.n_cols, 1, arma::fill::randn);
+
+  // Run the optimization
+  lbfgs.Optimize(lrf, beta);
+
+  return beta;
+}
+```
+
+### Verifying Results
+
+Prior to using the new optimizer in mission critical work, compare the results
+to methods already implemented in _R_. The best way to achieve this is to create
+an oracle model by specifying the parameters known to generate data and, then,
+try to recover them. Moreover, if a method is already implemented in _R_ feel
+free to try to check the result equality within an appropriate tolerance threshold.  
+
+Following with this methodology, data must be generated. 
+
+```{r data-generation}
+n <- 1e6
+beta <- c(-2, 1.5, 3, 8.2, 6.6)
+p <- length(beta)
+X <- cbind(1, matrix(rnorm(n), ncol = p - 1))
+y <- X %*% beta + rnorm(n / (p - 1))
+```
+
+Next, the optimization procedure is used to estimate the parameters of interest.
+Under this example, the results of the estimation can be compared to `lm()`. 
+That said, `lm()` may have more precise results when compared against the 
+optimizer as it is implemented with a closed-form solution to linear regression
+plus the computational is performed more rigorously.  
+
+```{r data-estimation, eval = FALSE}
+coefs_lbfgs <- lin_reg_lbfgs(X, y)
+coefs_lm <- lm.fit(X, y)$coefficients
+```
+
+```{r echo = FALSE}
+coefs_lbfgs <- RcppEnsmallen::lin_reg_lbfgs(X, y)
+coefs_lm <- lm.fit(X, y)$coefficients
+compare_coefs = cbind(coefs_lbfgs, coefs_lm)
+colnames(compare_coefs) = c("LBFGS", "LM")
+rownames(compare_coefs) = paste0("Beta", seq_len(nrow(compare_coefs)))
+knitr::kable(compare_coefs, longtable = FALSE, caption = "Comparison of Estimated Coefficients")
+```
+
+# Package Usage
+
+RcppEnsmallen is best used within an _R_ package. The setup for `RcppEnsmallen`'s
+use mirrors that of other `Rcpp`-based projects. In particular, the
+`DESCRIPTION` file requires the `LinkingTo` field to contain:
+
+```bash
+LinkingTo: Rcpp, RcppArmadillo (>= 0.8.400.0.0), RcppEnsmallen (>= 0.2.10.0.1)
+```
+
+Next, the `src/` directory must contain both a `Makevars` and `Makevars.win` 
+file with:
+
+```bash
+CXX_STD = CXX11
+PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS)
+PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)
+```
+
+The `Makevars.win` file provides the appropriate configuration for Windows
+while `Makevars` acts on Unix-alike systems like macOS, Linux, and Solaris.
+

Original file line number	Diff line number	Diff line change
`@@ -30,7 +30,7 @@ struct MatTypeTraits<arma::Col<eT>>`
`30`	`30`	`template<typename eT>`
`31`	`31`	`struct MatTypeTraits<arma::Row<eT>>`
`32`	`32`	`{`
`33`		`- typedef arma::Row<eT> BaseMatType;`
	`33`	`+ typedef arma::Mat<eT> BaseMatType;`
`34`	`34`	`};`
`35`	`35`
`36`	`36`	`template<typename eT>`