foeach()%dofuture% can't find .bat file

[Maize-Blue]

Trying to get %dofuture% to work, but getting an error using the following code. Not sure why.

library(foreach)
library(doParallel)
library(doFuture)

plan(multisession,workers=availableCores())

# Create a .bat file in your downloads folder called "Example1.bat" including the following:
# @ECHO OFF
# ECHO Such a useful reproducible code...

Folder.Path <- "C:/Users/Insert.Name.Here/Downloads/"
setwd(Folder.Path)
system2("Example1.bat") # this runs as expected
foreach(i=1)%do%{system2(paste0("Example",i,".bat"))} # this runs as expected
foreach(i=1)%dopar%{system2(paste0("Example",i,".bat"))} # this runs as expected
foreach(i=1)%dofuture%{system2(paste0("Example",i,".bat"))} # this returns a warning

This is the warning message I get when I use %dofuture%

Warning message:
In system2(paste0("Example", i, ".bat")) : '"Example1.bat"' not found

[HenrikBengtsson]

This happens because the working directory of the parallel workers does not change when you change the working directory in your main R session.

My recommendation is to avoid setwd() in your scripts, and instead work with the full pathnames, e.g.

Folder.Path <- "C:/Users/Insert.Name.Here/Downloads/"
res <- foreach(i = 1) %dofuture% {
  pathname <- file.path(Folder.Path, paste0("Example",i,".bat"))
  system2(pathname)
}

Personally, I always try to avoid indices (here i), so I would do something like:

library(doFuture)
Folder.Path <- "C:/Users/Insert.Name.Here/Downloads/"
pathnames <- file.path(Folder.Path, paste0("Example", 1:10, ".bat"))
res <- foreach(pathname = pathnames) %dofuture% {
  system2(pathname)
}

or

library(future.apply)
Folder.Path <- "C:/Users/Insert.Name.Here/Downloads/"
pathnames <- file.path(Folder.Path, paste0("Example", 1:10, ".bat"))
res <- lapply(pathnames, function(pathname) {
  system2(pathname)
})

BTW, when you use doFuture, there's no need for the doParallel package.

[Maize-Blue]

Thanks for the response! I definitely didn't account for the order of setwd() and plan(multisession)

Is there a reason you avoid indices in particular? I'm using them since I originally had for loops written before realizing I had to learn how foreach loops work for parallelization which work slightly different

Additionally, this example was the simplest reproducible example I could write. I'm actually running another program with .bat files serving as the medium between R and those programs so often I don't need an output.

[HenrikBengtsson]

Is there a reason you avoid indices in particular? I'm using them since I originally had for loops written before realizing I had to learn how foreach loops work for parallelization which work slightly different

Indices tricks you to stay away for lapply(), map(), etc. And, worse, it makes you think foreach() is replacing a for-loop. That's not true - foreach is not a for-loop!. Repeat that 100 times each time you go to bed until you know it by heart. foreach() is no different from lapply() and map(). See Slide 13 ("Let’s talk foreach") and beyond in https://henrikbengtsson.github.io/future-tutorial-raukr2024/BengtssonH_20240612-futureverse-part2-raukR_2024.pdf for details.

[Maize-Blue]

I tried using the first chunnk above. I'm not getting the desired output and res[[1]] is showing 0

library(foreach)
library(doFuture)

# Create a .bat file in your downloads folder called "Example1.bat" including the following:
# @ECHO OFF
# ECHO Such a useful reproducible code...

Folder.Path <- "C:/Users/Insert.Name.Here/Downloads/"
setwd("C:/")

# This prints what I want
system2(paste0(Folder.Path,"Example1.bat")) 

# Set up workers in C:/
plan(multisession,workers=availableCores())

# res[[1]] output is 0 in the output using the first provided chunk above
res <- foreach(i=1)%dofuture%{
          pathname <- file.path(Folder.Path,paste0("Example",i,".bat"))
          system2(pathname)
      }

[HenrikBengtsson]

I'm not getting the desired output ...

And what is that?

... and res[[1]] is showing 0

The last expression in your foreach() iterations is system2(...) and it's that value that you get. system2() returns an integer, which is zero if it succeeds. See ?system2. So, everything is as expected.