Future throws a warning that is invisible in a regular call

[Jean-Romain]

Hi,

My code throws warnings that are excessively hard to track because the warning are thrown only within future and the origin of the problem is lost. This issue arises when I initialize a RasterLayer with NAs

With a regular call this does not throw any warning:

r <- raster::raster()
r[] <- NA
r

However, once wrapped in future it does throw two warnings without trace

f = future::future(
{
  r = raster::raster()
  r[] <- NA
  r
})
future::value(f)
#> Messages d'avis :
#> 1: Dans min(x@data@values, na.rm = TRUE) :
#>   aucun argument trouvé pour min ; Inf est renvoyé
#> 2: Dans max(x@data@values, na.rm = TRUE) :
#>   aucun argument pour max ; -Inf est renvoyé

Interestingly, the warnings are valid and expected. They correspond to

min(rep(NA, 10), na.rm = TRUE)

I would like to know what settings I should tune in R to either:

• make future behaving like a regular call to get rid of this specific warning that is never raise out of this specific case AFAIK
• make the regular call behaving like future so I can track where the problem come from and fix it.

Thanks

[HenrikBengtsson]

It looks like this warning is coming from setMinMax() for RasterLayer;

	if ( inMemory(x) ) {
		x@data@min <- min(x@data@values, na.rm=TRUE)
		x@data@max <- max(x@data@values, na.rm=TRUE)
	} else {

I suspect it is because that r[] <- NA call ends up taking different paths in the raster code depending on it running in the main R session or in a background worker. Now, exactly why that happens, I'm not sure, and it would require more delicate understanding on the inner of raster than I possess. FWIW, I'd believe it's possible to reproduce this warning using only raster itself, i.e. without parallel/future processing. May @rhijmans can clarify what's going on and how to avoid these warnings.

However, I agree with you, it seems like

r <- raster::raster()
r[] <- NA

should also produce a warning here. That's something raster need to solve.

[rhijmans]

The design is for raster not to show a warning with

r[] <- NA

Because this is what setMinMax starts out with

w <- getOption('warn')
on.exit(options('warn' = w))
options('warn'=-1) 
min(x@data@values, na.rm=TRUE)

Apparently that does not work with future --- perhaps because things happen in a different environment?
I have changed raster to use

suppressWarnings(min(x@data@values, na.rm=TRUE))

And now the warnings are not produced with future

[HenrikBengtsson]

The design is for raster not to show a warning with

r[] <- NA

Because this is what setMinMax starts out with

w <- getOption('warn')
on.exit(options('warn' = w))
options('warn'=-1) 
min(x@data@values, na.rm=TRUE)

Apparently that does not work with future --- perhaps because things happen in a different environment? I have changed raster to use

suppressWarnings(min(x@data@values, na.rm=TRUE))it exists

And now the warnings are not produced with future

Thank you for the quick explanation and fix.

This is an interesting use case for the warn option, which may illustrate how and where it's meant to be used. In the 'R Language Definition' (https://cran.r-project.org/doc/manuals/R-lang.html#warning), we can read:

8.2 warning

The function warning takes a single argument that is a character string. The behaviour of a call to warning depends on the value of the option "warn". If "warn" is negative warnings are ignored. If it is zero, they are stored and printed after the top-level function has completed. If it is one, they are printed as they occur and if it is 2 (or larger) warnings are turned into errors.

If "warn" is zero (the default), a variable last.warning is created and the messages associated with each call to warning are stored, sequentially, in this vector. If there are fewer than 10 warnings they are printed after the function has finished evaluating. If there are more than 10 then a message indicating how many warnings occurred is printed. In either case last.warning contains the vector of messages, and warnings provides a way to access and print it.

and in ?options, we can read something very similar:

warn:
integer value to set the handling of warning messages. If warn is negative all warnings are ignored. If warn is zero (the default) warnings are stored until the top–level function returns. … If warn is one, warnings are printed as they occur. If warn is two (or larger, coercible to integer), all warnings are turned into errors.

Now, I actually don't know the original intention, but my best guess is that warn is meant to control what happens to warnings if they bubble up all the way to the top-level, i.e. if the warning is not caught and handled before that (e.g. suppressWarnings()), then R looks at getOption("warn") to decided what to do. I think "If "warn" is negative warnings are ignored" above should be read that way, and not what action is taken when the warning is produced (e.g. by warning()), which the above "The behaviour of a call to warning depends on the value of the option "warn"" actually suggests.

Continuing, a future "subscribes" to all conditions, by evaluating the R expression within a withCallingHandlers(expr, conditions = function(cond) ...). This is what records those warnings, despite getOption("warn") < 0. Thus, options(warn = -1) has no effect because when the captured warning condition it's relayed in the main R session, it'll bubble up to the top-level and be signalled according to whatever warn is set there (default to warn = 0).

Here's an example of what's going on using plain, sequential R:

$ R --quiet --vanilla

options(warn = -1)

w <- NULL

res <- withCallingHandlers({
  warning("boom"); 42L
}, warning = function(cond) w <<- cond)

print(res)
#> [1] 42

str(w)
#> List of 2
#>  $ message: chr "boom"
#>  $ call   : language withCallingHandlers({     warning("boom") ...
#>  - attr(*, "class")= chr [1:3] "simpleWarning" "warning" "condition"

By you now using suppressWarnings(), it'll never reach the future in the first place, e.g.

w <- NULL

res <- withCallingHandlers(suppressWarnings({
  warning("boom"); 42L
}), warning = function(cond) w <<- cond)

print(res)
#> [1] 42

str(w)
#> NULL

TIL, or possible TIRL (=relearned).

[Jean-Romain]

That makes sense. Thank you. Indeed in this specific case the use of suppressWarning is the good way to handle the problem and changing the warn options does not fit with the intention of the option. @rhijmans fixed it and I will try use withCallingHandlers to track my problem. Thank you

[HenrikBengtsson]

FWIW, it looks like it's rather common among CRAN packages to set option warn = -1 (https://github.com/search?q=org%3Acran+language%3Ar+R%2F+in%3Afile%2Cpath+options+warn+%22-1%22&type=Code)

[rhijmans]

CRAN tells you not to do that when you submit a new package (that is how I found out about suppressWarnings) --- but I think that is a new thing; and R CMD check does not complain if you add it to updates.

[Jean-Romain]

Thanks @rhijmans this will help. I usually wrap r[] <- NA in suppressWarnings myself to avoid this problem. But I missed one somewhere and because my code is a complex stack of functions I cannot find where. This was the point of my question. I'd like to be able to reproduce what future does to either get rid of the warning by setting proper options or to reproduce the warning outside future to figure out where I missed a suppressWarning

[rhijmans]

I know this is tangential, but I am surprised you use r[] <- NA at all --- but perhaps there is a context where that is necessary.

[Jean-Romain]

Do you suggest another syntax? The typical use case is when I need to initialize a template raster that will be filled later. Initializing with raster:raster() returns a RasterLayer with 0 data (logical(0)). And for some reason (I do no exactly remember why) I need this raster to have data so I initialize it with NAs.

[rhijmans]

I just could not think of a reason to do that; my intent is that it would not be necessary. For example you can do r=raster(ncol=5, nrow=5); r[4]=10; but there might well be some cases where it is necessary or efficient.