Skip to content

Regression with Armadillo 15.0.0 -- and 14.6.* under Intel #474

@eddelbuettel

Description

@eddelbuettel

The eBsc package package is experiencing a very strange regression when going from Armadillo 14.6.* (either .0 or .3) to 15.0.0. So this looks like a change to worse. Oddly, the same error also appears under the Intel compiler where (copying here in case the link disappears) we currently see

* checking examples ... [9s/11s] ERROR
Running examples in ‘eBsc-Ex.R’ failed
The error most likely occurred in:

> ### Name: plot.eBsc
> ### Title: Plot fitted components
> ### Aliases: plot.eBsc
> ### Keywords: plot
> 
> ### ** Examples
> 
> 
> library(eBsc)
> n <- 250
> sigma <- 0.05
> Basis <- list()
> for(i in 1:6) Basis[[i]] <- drbasis(nn = n, qq = i)
Error: rank(): failed
Execution halted

Here the example for plot.eBsc fails with insuffcient rank(). We can trace this back to (R function) drbasis() which calls C++ function drbasis(). For arguments nn=250, qq=5 this fails deterministically.

We isolated the core of the qq=5 code (which is rather neatly inside a case statement) having it return the matrix for which the rank() fails, and have it print the final ten column of the first ten rows. The resulting C++ file has zero dependencies on R or R packages. No changes were made apart from reindentation, and removing of extra code not needed for a minimally viable replicable example as provided here.

For convenience we just point at the include directories provided by what is currently PR #473 which has both Armadillo 15.0.0 (for compilations under C++14 or newer) and Armadillo 14.6.3 (for C++11 compilations as more than two hundred CRAN packages still hardcode).

Note that the following example can be reproduced anywhere with both Armadillo 15.0.0 or equivalent, and the older Armadillo 14.6.* by adjusting the -I... location. current is the released upstream version 15.0.0, legacy the preceding 14.6.3. We do not need to link with -larmadillo or -lblas or -llapack etc as we execute no such functionality as only vector / matrix code is used.

edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ g++ -Wall -pedantic \   # indented for display
    -I/usr/local/lib/R/site-library/RcppArmadillo/include/legacy  \
    -o ebsc_rank_issue_main ebsc_rank_issue_main.cpp -lm 
edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ ./ebsc_rank_issue_main 
M[0:9, 240:249]
   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000
   0.0698   0.0704   0.0710   0.0717   0.0722   0.0728   0.0734   0.0739   0.0744   0.0749
  -0.0796  -0.0804  -0.0811  -0.0818  -0.0823  -0.0829  -0.0834  -0.0838  -0.0841  -0.0844
   0.0803   0.0816   0.0829   0.0840   0.0849   0.0858   0.0865   0.0871   0.0876   0.0879
  -0.0759  -0.0781  -0.0802  -0.0821  -0.0838  -0.0852  -0.0865  -0.0875  -0.0883  -0.0888
   0.0688   0.0722   0.0754   0.0783   0.0809   0.0831   0.0850   0.0866   0.0878   0.0887
  -0.0602  -0.0650  -0.0695  -0.0735  -0.0772  -0.0804  -0.0831  -0.0854  -0.0871  -0.0884
   0.0505   0.0568   0.0626   0.0680   0.0729   0.0772   0.0809   0.0839   0.0863   0.0881
  -0.0397  -0.0476  -0.0550  -0.0618  -0.0680  -0.0735  -0.0783  -0.0823  -0.0854  -0.0876
   0.0283   0.0377   0.0467   0.0550   0.0626   0.0695   0.0754   0.0804   0.0843   0.0871
edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ g++ -Wall -pedantic \   # indented for display
    -I/usr/local/lib/R/site-library/RcppArmadillo/include/current  \
    -o ebsc_rank_issue_main ebsc_rank_issue_main.cpp -lm 
edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ ./ebsc_rank_issue_main 
M[0:9, 240:249]
      nan      nan      nan      nan      nan      nan      nan      nan      nan      nan
   0.0698      nan      nan      nan      nan      nan      nan      nan      nan      nan
  -0.0796  -0.0804      nan      nan      nan      nan      nan      nan      nan      nan
   0.0803   0.0816   0.0829      nan      nan      nan      nan      nan      nan      nan
  -0.0759  -0.0781  -0.0802  -0.0821      nan      nan      nan      nan      nan      nan
   0.0688   0.0722   0.0754   0.0783   0.0809      nan      nan      nan      nan      nan
  -0.0602  -0.0650  -0.0695  -0.0735  -0.0772  -0.0804      nan      nan      nan      nan
   0.0505   0.0568   0.0626   0.0680   0.0729   0.0772   0.0809      nan      nan      nan
  -0.0397  -0.0476  -0.0550  -0.0618  -0.0680  -0.0735  -0.0783  -0.0823      nan      nan
   0.0283   0.0377   0.0467   0.0550   0.0626   0.0695   0.0754   0.0804   0.0843      nan
edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ 

The self-contained C++ code is below.

// The eBsc package on CRAN (at version 4.17) reports an error under the Intel compiler.
// This error now also appears with RcppArmadillo 15.0.0. In the example for plot.eBsc,
// the function drbasis() is called with n=250 and a second parameter p that varies from
// 1 to 6. In the case of 5, NA values return leading an rank failure. The drbasis()
// function calls an underlying C function drbasis from file drbasisC.cpp which has case
// statements for the differenct values of p.  We extrace the one for 5 here.

#include <armadillo>

arma::mat compute(int nn) {
    using namespace arma;
    using namespace std;
    const double E = exp(1);
    arma::vec t(nn);
    arma::vec k(nn);
    typedef complex<double> dcomp;
    double Pi = M_PI;
    complex<double> comp0 (0,1);
    complex<double> comp1 (-1,0);
    complex<double> comp2 (0,-0.5);
    complex<double> comp3 (1,-1);
    complex<double> comp4 (-1,1);
    complex<double> comp5 (0,0.5);
    complex<double> comp6 (0,0.25);
    int const0 = 1;
    double const1 = -sqrt(3);
    double const2 = 2*sqrt(3);
    double const3 = -sqrt(5);
    double const4 = 6*sqrt(5);
    double const5 = -6*sqrt(5);
    double const6 = -sqrt(7);
    double const7 = 12*sqrt(7);
    double const8 = -30*sqrt(7);
    double const9 = 20*sqrt(7);
    double const10 = 3;
    double const11 = -60;
    double const12 = 270;
    double const13 = -420;
    double const14 = 210;
    /*
    double const15 = -sqrt(11);
    double const16 = 30*sqrt(11);
    double const17 = -210*sqrt(11);
    double const18 = 560*sqrt(11);
    double const19 = 630*sqrt(11);
    double const20 = 252*sqrt(11);
    */

    for (int j=0;j<nn;j++) {
        t(j)= j/double((nn-1));
    }
    for (int j=0;j<nn;j++) {
        k(j)= j+1;
    }

    arma::mat M5(nn,nn);
    for (int i=0;i<nn; i++) {
        for (int j=0;j<nn;j++) {
            if (j==0) {
                M5(i,j)=const0;
            } else if (j==1) {
                M5(i,j)= const1 + const2*t(i);
            } else if (j==2) {
                M5(i,j)= const3 + const4*t(i) + const5* pow(t(i),2);
            } else if (j==3) {
                M5(i,j)= const6 + const7*t(i) + const8* pow(t(i),2) + const9* pow(t(i),3);
            } else if (j==4) {
                M5(i,j)= const10 + const11*t(i) + const12* pow(t(i),2) + const13* pow(t(i),3) + const14* pow(t(i),4);
            } else {
                dcomp ans = (sqrt(2)*(1 + sqrt(5)/2) -
                             (comp5*(sqrt(10 - 2*sqrt(5)) +
                                     sqrt(2*(5 + sqrt(5)))))/sqrt(2))*(pow(comp1,1 + k(j))/pow(E,pow(comp1,0.1)* (dcomp(-3 + k(j)))*Pi*(1 - t(i))) + pow(E,-(pow(comp1,0.1)*(dcomp(-3 + k(j)))* Pi*t(i)))) +
                    (-(1/sqrt(2)) -
                     (comp5*(sqrt(10 - 2*sqrt(5)) + sqrt(2*(5 + sqrt(5)))))/
                     sqrt(2))*(pow(comp1,1 + k(j))/pow(E,pow(comp1,0.3)*(dcomp (-3 + k(j)))*Pi*(1 - t(i))) + pow(E,-(pow(comp1,0.3)*(dcomp (-3 + k(j)))*Pi*t(i)))) +
                    (sqrt(2)*(1 + sqrt(5)/2) + (comp5*(sqrt(10 - 2*sqrt(5)) + sqrt(2*(5 + sqrt(5)))))/sqrt(2))
                    *(pow(comp1,1 + k(j))/ pow(E,(sqrt(0.625 + sqrt(5)/8) - comp6*(-1 + sqrt(5)))*(dcomp (-3 + k(j)))*Pi*
                                               (1 - t(i))) +pow(E,-((sqrt(0.625 + sqrt(5)/8.) - comp6*(-1 + sqrt(5)))*(dcomp (-3 + k(j)))*
                                                                    Pi*t(i)))) +(-(1/sqrt(2)) + (comp5*(sqrt(10 - 2*sqrt(5)) + sqrt(2*(5 + sqrt(5)))))/sqrt(2))
                    *(pow(comp1,1 + k(j))/pow(E,(dcomp(sqrt(0.625 - sqrt(5)/8)) - comp6*(1 + sqrt(5)))*
                                              (dcomp (-3 + k(j)))*Pi*(1 - t(i))) +pow(E,-((sqrt(0.625 - sqrt(5)/8) - comp6*(1 + sqrt(5)))* (dcomp (-3 + k(j)))*Pi*t(i))))- sqrt(2)*cos((-3 + k(j))*Pi*t(i));
                M5(i,j) =ans.real();
            }
            M5(i,j) = M5(i,j)/sqrt(nn);
        }
    }
    return M5;
}

int main(int argc, char *argv[]) {
    constexpr int nn = 250;
    arma::mat M = compute(nn);
    M.submat(0, 240, 9, 249).print("M[0:9, 240:249]");
    exit(0);
}

I have not found any compiler switches, or compilation standard changes, affecting this. As the result also appears under the Intel compiler for the older Armadillo, I suspect the package has code with a latent frailty revealed by either the Intel compiler, or the newer Armadillo. The eBsc maintainer has been contacted by email a few days ago, I have yet to hear back.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions