Progress bar on remote slurm cluster using future.batchtools

[ARLundborg]

Below is a modified and working example of how to use a progress bar on a cluster. It works great!

library(parallelly)
library(parallel)
library(future)
library(future.apply)
library(progressr)

### Initialise progress bar
handlers(handler_progress(format="[:bar] :percent :eta :message"))
handlers(global=TRUE)


### Initialise computing cluster
worker_names <- sapply(1:10, function(s) paste0("worker-", s)) 
nodes_per_worker <- 2

my_cluster <- ### Setup a cluster using makeClusterPSOCK
plan(cluster, workers=my_cluster)


### Define simulations
reps <- 10
sim_df <- data.frame(rep=1:reps)

compute_func <- function(sim_df) {
  prog_bar <- progressor(along=1:(nrow(sim_df)))
  sim_res <- future_apply(sim_df, MARGIN=1, future.seed=TRUE, simplify=FALSE, FUN = function(x) {
    rep <- x["rep"]
    Sys.sleep(1)
    return(rep^2)
})}

sim_res <- compute_func(sim_df)

stopCluster(my_cluster)

Now, instead of a cluster, I have access to a remote HPC cluster using slurm and I do the following.

library(parallelly)
library(parallel)
library(future)
library(future.apply)
library(progressr)

### Initialise progress bar
handlers(handler_progress(format="[:bar] :percent :eta :message"))
handlers(global=TRUE)


### Initialise remote HPC cluster
login <- tweak(remote, workers="worker-name")

slurm <- tweak(batchtools_slurm, template=quote(system.file("templates/slurm-simple.tmpl", package="batchtools")), resources=list(walltime=60*60*24, ncpus=2))

plan(list(login, slurm))


### Define simulations
reps <- 10
sim_df <- data.frame(rep=1:reps)

compute_func <- function(sim_df) {
  prog_bar <- progressor(along=1:(nrow(sim_df)))
  sim_res <- future_apply(sim_df, MARGIN=1, future.seed=TRUE, simplify=FALSE, FUN = function(x) {
    rep <- x["rep"]
    Sys.sleep(1)
    return(rep^2)
})}

sim_res %<-% compute_func(sim_df)

This works in the sense that the slurm cluster is returning the values but two related things are bothering me.

1. The progress bar is not present at all. I'm not sure how to modify the code to make the progress bar work?
2. In the first example, my R process is locked and working while printing the progress bar for the duration of the run time of cluster_func. In the second example the R process instantly finishes unless I try to access the sim_res variable. I'm aware that this is intentional (and I'm sure incredibly handy at time) I would rather the process locked and waited like in the first example. Is there a way of achieving this?

Thanks!

[HenrikBengtsson]

1. None of the future.batchtools backends are among the future backend that are capable of relaying progressr updates, cf. https://progressr.futureverse.org/articles/progressr-intro.html#near-live-versus-buffered-progress-updates-with-futures.

2. As you say, that is the design, so you need to access sim_res.