Improve seq() consistency between integer/integer64

[MichaelChirico]

I think this is not desirable:

seq(as.integer64(1), 10, by=1.5)
# integer64
#  [1] 1  2  3  4  5  6  7  8  9  10
seq(1, 10, by=1.5)
# [1]  1.0  2.5  4.0  5.5  7.0  8.5 10.0

The trouble is that we can't always embed into double from integer64:

seq(as.integer64(2^60), by=1L, length.out=10L)
# integer64
#  [1] 1152921504606846976 1152921504606846977 1152921504606846978
#  [4] 1152921504606846979 1152921504606846980 1152921504606846981
#  [7] 1152921504606846982 1152921504606846983 1152921504606846984
# [10] 1152921504606846985

sprintf("%18.f", seq(2^60, by=1L, length.out=10L))
#  [1] "1152921504606846976" "1152921504606846976" "1152921504606846976"
#  [4] "1152921504606846976" "1152921504606846976" "1152921504606846976"
#  [7] "1152921504606846976" "1152921504606846976" "1152921504606846976"
# [10] "1152921504606846976"

That said, the seq() output is entirely determined by from, to, and by; we can cheaply examine these up front and determine if the output sequence embeds entirely in (1) integer64; (2) double; (3) neither.

Then we should determine the output as follows:

1. Return integer64
2. Return double
3. Warn, and return integer64

[MichaelChirico]

Ultimately, this is hard to distinguish from a debate over what basic mixed arithmetic like i64 + dbl should do. Currently, the package elects against embedding into a storage type for a given operation that would allow it to succeed with full fidelity, in favor of having predictable output types given input. There are tradeoffs here.

[hcirellu]

As for seq(as.integer64(1), 10, by=1.5) I would expect a double output consistent to seq(1L, 10, by=1.5).
Only with seq(1L, 10L, 1L) an integer output is returned. As soon as one of the arguments (from, to, by) is double, double is returned. I would prefer that seq(as.integer64(1), 10, by=1) returns double und if the output is not entirely determined a warning is displayed.

Some (theoretical) examples:

• seq(as.integer64(1), 10, by=1) returns num [1:10] 1 2 ... 10
• seq(as.integer64(1), 10L, by=1L) returns int64 [1:10] 1 2 ... 10
• seq(as.integer64(1), lim.integer64()[2], by=1L) returns int64 [1:9223372036854775807] 1 2 ... 9223372036854775807
• seq(as.integer64(1), lim.integer64()[2], by=1) returns num [1:9223372036854775807] 1 2 ... 9.223372e+18 with overflow warning

Other thoughts on that?

[MichaelChirico]

The problem with seq(as.integer64(1), 10, by=1.5) returning double is that while it's consistent with seq(1, 10, by=1.5), it is inconsistent within bit64 itself:

as.integer64(1) + (0:6) * 1.5
# integer64
# [1] 1  2  4  5  7  8  10

So we may have opened a can of worms -- should all operations check if the output fully embeds in a lower type? That is messing around with some of the fundamental wiring/design choices of the package. And the performance implications for that are more serious -- every input element, in principle, needs to be checked (whereas for seq(), we can figure it out entirely from the endpoints + increment, much faster).

If there were a widely-available 128-byte float type, I would say -- fine, just coerce to a space that has full support for both integer64 and double, no risk of information loss.

What we may be stuck with as a way forward is a new option to the generic, like smart_type, which when activated, tries to be "smart" about the type as we laid out above, but keeps back-compatibility by default.

[hcirellu]

My example of thinking about the general behaviour is c(as.integer64(1L), 0.25, 1e+300). I would have expected the return value is double. This is mainly because I understand integer64 as an extended integer and not a special data type like POSXlt. In addition, (almost) everytime I mix integer and double the return value is double.
This comfortable automatic data type conversion reduces performance (if the conversion is not intended). So when it comes to performance relevant parts, I am more precise with choosing and assuring the data type during coding, so that no unnecessary data type conversion takes place.

I would not introduce a "smart" detection in a way that if it fits in integer64 an integer64 is returned and otherwise not. Because then, the result is value dependent and the code is possibly harder to debug and profile. I would prefer a fixed setting like prefer_double_over_integer64, such that in my example if set to FALSE integer64 is returned (actual behavior) and if set to TRUE double is returned. This could hopefully be integrated easier and with less performance loss.

if (isTRUE(getOption("bit64.prefer_double_over_integer64", FALSE))) {
  c(as.integer64(1L), 0.25, 1e+300) ==> num [1:3] 1.0 2.5e-01 1.0e+300
} else {
  c(as.integer64(1L), 0.25, 1e+300) ==> integer64 [1:3] 1 0 NA  (with a int64 overflow warning)
}