R Project Sprint 2023 - Tweak printCoefmat() - better Round()ing / Zapping Zeros

[giscus[bot]]

R Project Sprint 2023 - Tweak printCoefmat() - better Round()ing / Zapping Zeros

In rare cases, the rounding, e.g. of the ‘Std. Error’. column in printCoefmat() is undesirably rounding too much and then depicting inaccurate values. This is from the recent R-devel mailing list thread starting at https://stat.ethz.ch/pipermail/r-help/2023-July/477688.html

https://contributor.r-project.org/r-project-sprint-2023/projects/tweak-printCoefmat/

[trangdata]

A quick question: Would showing the current desired output be misleading? Specifically, if I look at the table below, I would think that the intercept is less than 1e-06 (which it actually is in this case, but zapsmall in this case would zap anything less than 1e-04):

            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 0.000000   0.001115     0.0        1    
x1          0.499263   0.001128   442.4   <2e-16 ***
x2          0.600990   0.001108   542.3   <2e-16 ***

Looking at the source code, I don't see an easy way to conform the two steps of rounding with different digits (first zap.ind and then cs.ind), but I wonder if we should leave other cs.ind columns as are and only show rounding of the zap.ind column?

          Estimate Std. Error t value Pr(>|t|)    
(Intercept) 0.0000  1.115e-03     0.0        1    
x1          0.4992  1.128e-03   442.4   <2e-16 ***
x2          0.6010  1.108e-03   542.3   <2e-16 ***

[mmaechler]

Sorry I saw this only now. Yes, you are right here. OTOH, if we look at the table the original idea was that Estimate and Std. Error should use the same format (at least in some sense) so you could easily do approximate beta +/- c * se (c = 1 or 2, say) for approximate confidence intervals in your head.
If one uses scientific/exponential notation and the other not, or for fixed notation, the number of digits after the decimal point differs, such appproximate computations in your head become less easy.

There's another point, Thomas Lumley recently made (on Mastodon): The precision that makes sense for the SE is typically really small, IIRC, he said 2 (significan!) digits should always suffice, and that order of magnitude is definitely correct.
So for the example we have here, it would make sense to show 'Std. Error' in fixed format with only 4 digits after the decimal, the same number you show above for the Estimate.

OTOH: In an exam recently (opening easy question) we asked the students to compute the t-value (we replaced it with "???" in the R output). If they had only 2 digits for the S.E., the t-value also only gets 2 signfiicant digits ... which in this case, where you really like to show something 442-like, and not 4.42e+03 ...

[trangdata]

Great point!

So to keep the consistency between the two first columns and also across zap.ind options, do you think we should remove the computation of digmin (essentially digmin = 0)? And the user can control the number of significant figures to be printed with the digits argument?

https://github.com/r-devel/r-svn/blob/236f2202c3ffb739fbd2b99bdb6c2670c00ce53b/src/library/stats/R/anova.R#L112-L114

[trangdata]

@mmaechler if you would entertain my idea a little further, this is what we would see if digmin = 0:

options(digits = 6)
printCoefmat(sfm$coefficients, zap.ind = 1)
#>             Estimate Std. Error t value Pr(>|t|)    
#> (Intercept)   0.0000     0.0011     0.0        1    
#> x1            0.4992     0.0011   442.4   <2e-16 ***
#> x2            0.6010     0.0011   542.3   <2e-16 ***
options(digits = 7)
printCoefmat(sfm$coefficients, zap.ind = 1)
#>             Estimate Std. Error  t value Pr(>|t|)    
#> (Intercept)  0.00000    0.00112   0.0003   0.9997    
#> x1           0.49923    0.00113 442.4335   <2e-16 ***
#> x2           0.60099    0.00111 542.2768   <2e-16 ***

A few things I want to note for myself:

• digits have different default values in print.summary.lm and printCoefmat (max(3L, getOption("digits") - 3L) vs. max(3L, getOption("digits") - 2L)).

• If I increase the number of digits from the default of 4 to 5, print(sfm, zap.ind = 1, digits = 5), I see a "jump" in the number of decimal places in the t value column. This is because format is now rounding the intercept of 0.0003, not 0.000 like before.,

             Estimate Std. Error  t value Pr(>|t|)    
(Intercept) 0.0000000  0.0011153   0.0003   0.9997    
x1          0.4992300  0.0011284 442.4335   <2e-16 ***
x2          0.6009900  0.0011083 542.2768   <2e-16 ***