future.seed

[markusloecher]

I find the results of this testing script regarding future.seed a bit strange:
y1 is equal to y0 which shows that even with future.seed =F everything works well w.r.t. the seed
But the package throws a strong warning.
y2 is totally different which worries me but then again it is also using a different seed each time.
I would conclude from this to ignore the stern warning and set future.seed=F in most cases ? But I am certainly missing sth. ?

dice = function(seed) {
  set.seed(seed)
  d = sample(6,3)
  return(d)
}

y0 = lapply(1:4, FUN = dice)
pint(y0)

y1 <- future_lapply(1:4, FUN = dice)
print(y1)

y2 <- future_lapply(1:4, FUN = dice, future.seed = TRUE)
print(y2)

[HenrikBengtsson]

Hello. The reason for seed = TRUE (y2) being different is that it uses a random number generator (RNG) that is designed for parallel processes - we need to use a parallel RNG in order to get statistically sound random numbers. With the default seed = FALSE (y1) we're using the default sequential RNG that exist on each parallel worker independently of each other. That may or may not be okay to do, depending on what you use the random numbers for. But the safest is to use a proper parallel RNG.

set.seed(42) will set the RNG seed in whatever the current "RNG kind" is. The RNG kind is what differs between sequential and parallel RNGs. The default in R is sequential RNG of kind Mersenne-Twister, whereas most parallel framworks, including Futureverse, uses L'Ecuyer-CMRG. So, even with the same set.seed(n), these produce different random numbers, e.g.

> RNGkind()
[1] "Mersenne-Twister" "Inversion"        "Rejection" 
> set.seed(42)
> rnorm(3)
[1]  1.3709584 -0.5646982  0.3631284

and

[1] 1.3709584 -0.5646982 0.3631284

> RNGkind("L'Ecuyer-CMRG")
> RNGkind()
[1] "L'Ecuyer-CMRG" "Inversion"     "Rejection"    
> set.seed(42)
> rnorm(3)
[1] -0.93907708 -0.04167943  0.82941349

See also #764, which has a discussion around this.

FWIW, I would refrain from using set.seed(w_i) in functions, especially if it used to give parallel workers different RNG streams. It's common to see, but it's a hack, and it's better to use a proper RNG such as L'Ecuyer's CMRG.

Hope this helps