PERFORMANCE: result() temporarily sets 'fs.latency = 1.0' second when calling batchtools::getLog()

Author: HenrikBengtsson

DESCRIPTION

@@ -1,5 +1,5 @@
 Package: future.batchtools
-Version: 0.10.0-9027
+Version: 0.10.0-9028
 Depends:
     R (>= 3.2.0),
     parallelly,

NEWS.md

@@ -43,6 +43,19 @@
 
  * Add a template file for SGE.
 
+## Miscellaneous
+
+ * `result()` for HPC batchtools backends would use a timeout of
+   `fs.latency` seconds (as set for the cluster functions) when trying
+   to collect the logged output.  However, since it has already
+   collected the results, the log file should be available already and
+   there would be no need to have to wait for the log file to appear.
+   Because of this, we temporarily set `fs.latency = 1.0` (second)
+   timeout for trying to find the log file.  This makes a big
+   difference in case the template used a `--output=<path>` location
+   other than `--output=<%= log.file %>`l in such cases the log file
+   would not be found, requiring a timeout.
+
 ## Bug Fixes
 
  * Using `plan(batchtools_nnn, finalize = FALSE)` would give a warning

R/BatchtoolsFuture-class.R

@@ -580,6 +580,10 @@ await <- function(future, cleanup = TRUE,
       if (inherits(result, "FutureResult")) {
         prototype_fields <- c(prototype_fields, "batchtools_log")
         result[["batchtools_log"]] <- try({
+	  ## Since we're already collected the results, the log file
+	  ## should already exist, if it exists.  Because of this,
+	  ## only poll for the log file for a second before giving up.
+	  reg$cluster.functions$fs.latency <- 1.0
           mdebug("- batchtools::getLog() ...")
           on.exit(mdebug("- batchtools::getLog() ... done"))
           getLog(id = jobid, reg = reg)