README: full revamp

Author: HenrikBengtsson

DESCRIPTION

@@ -1,5 +1,5 @@
 Package: future.batchtools
-Version: 0.12.2-9967
+Version: 0.12.2-9968
 Depends:
     R (>= 3.2.0),
     parallelly,

README.md

@@ -6,6 +6,39 @@
 
 # future.batchtools: A Future API for Parallel and Distributed Processing using 'batchtools' 
 
+## TL;DR
+
+Here is an example on how evaluate R expression on a Slurm
+high-performance compute (HPC) cluster.
+
+```r
+library(future)
+
+# Limit runtime to 10 minutes and memory to 400 MiB per future,
+# request a parallel environment with four slots on a single host.
+# On this system, R is available via environment module 'r'. By
+# specifying 'r/4.5.1', 'module load r/4.5.1' will be added to
+# the submitted job script.
+plan(future.batchtools::batchtools_slurm, resources = list(
+  time = "00:10:00", mem = "400M", nodes=1, ntasks=4,
+  modules = c("r/4.5.1")
+))
+
+# Give it a spin
+f <- future({
+  data.frame(
+    hostname = Sys.info()[["nodename"]],
+          os = Sys.info()[["sysname"]],
+       cores = unname(parallelly::availableCores()),
+     modules = Sys.getenv("LOADEDMODULES")
+  )
+})
+info <- value(f)
+print(info)
+#>   hostname    os cores  modules
+#> 1      n12 Linux     4  r/4.5.1
+```
+
 ## Introduction
 
 The **[future]** package provides a generic API for using futures in
@@ -24,93 +57,50 @@ high-performance computing (HPC) clusters via a simple switch in
 settings - without having to change any code at all.
 
 For instance, if **batchtools** is properly configured, the below two
-expressions for futures `x` and `y` will be processed on two different
-compute nodes:
+expressions for two futures will be processed on two different compute
+nodes:
 
 ```r
 library(future)
 plan(future.batchtools::batchtools_slurm)
 
-x %<-% { Sys.sleep(5); 3.14 }
-y %<-% { Sys.sleep(5); 2.71 }
+f_x <- future({ Sys.sleep(5); 3.14 })
+f_y <- future({ Sys.sleep(5); 2.71 })
+x <- value(f_x)
+y <- value(f_y)
 x + y
 #> [1] 5.85
 ```
 
 This is just a toy example to illustrate what futures look like and
 how to work with them.
 
-A more realistic example comes from the field of cancer research
-where very large data FASTQ files, which hold a large number of short
-DNA sequence reads, are produced.  The first step toward a biological
-interpretation of these data is to align the reads in each sample
-(one FASTQ file) toward the human genome.  In order to speed this up,
-we can have each file be processed by a separate compute node and each
-node we can use 24 parallel processes such that each process aligns a
-separate chromosome.  Here is an outline of how this nested parallelism
-could be implemented using futures.
-
-```r
-library(future)
-library(listenv)
-
-## The first level of futures should be submitted to the
-## cluster using batchtools.  The second level of futures
-## should be using multisession, where the number of
-## parallel processes is automatically decided based on
-## what the cluster grants to each compute node.
-plan(list(future.batchtools::batchtools_slurm, multisession))
-
-## Find all samples (one FASTQ file per sample)
-fqs <- dir(pattern = "[.]fastq$")
-
-## The aligned results are stored in BAM files
-bams <- listenv()
-
-## For all samples (FASTQ files) ...
-for (ss in seq_along(fqs)) {
-  fq <- fqs[ss]
-
-  ## ... use futures to align them ...
-  bams[[ss]] %<-% {
-    bams_ss <- listenv()
-	## ... and for each FASTQ file use a second layer
-	## of futures to align the individual chromosomes
-    for (cc in 1:24) {
-      bams_ss[[cc]] %<-% htseq::align(fq, chr = cc)
-    }
-	## Resolve the "chromosome" futures and return as a list
-    as.list(bams_ss)
-  }
-}
-## Resolve the "sample" futures and return as a list
-bams <- as.list(bams)
-```
+For an introduction as well as full details on how to use futures,
+please see <https://www.futureverse.org> or consult the package
+vignettes of the **[future]** package.
 
-Note that a user who do not have access to a cluster could use the
-same script processing samples sequentially and chromosomes in
-parallel on a single machine using:
 
-```r
-plan(list(sequential, multisession))
-```
+## Demos
 
-or samples in parallel and chromosomes sequentially using:
+The **[future]** package provides a demo using futures for calculating
+a set of Mandelbrot planes.  The demo does not assume anything about
+what type of futures are used.  _The user has full control of how
+futures are evaluated_.  For instance, to use local batchtools
+futures, run the demo as:
 
 ```r
-plan(list(multisession, sequential))
+library(future)
+plan(future.batchtools::batchtools_local)
+demo("mandelbrot", package = "future", ask = FALSE)
 ```
 
-For an introduction as well as full details on how to use futures,
-please consult the package vignettes of the **[future]** package.
-
-
 
-## Choosing batchtools backend
+## Available batchtools backend
 
 The **future.batchtools** package implements a generic future wrapper
 for all batchtools backends.  Below are the most common types of
-batchtools backends.
+batchtools backends. For other types of parallel and distributed
+backends, please see <https://www.futureverse.org/backends.html>.
 
 
 | Backend                  | Description                                                              | Alternative in future package
@@ -125,64 +115,16 @@ batchtools backends.
 | `batchtools_local`       | sequential evaluation in a separate R process (on current machine)       | `plan(cluster, workers = I(1))`
 
 
-### Examples
-
-Below is an examples on how use resolve futures via a Slurm scheduler.
-
-```r
-library(future)
-
-# Limit runtime to 10 minutes and memory to 400 MiB per future,
-# request a parallel environment with four slots on a single host.
-# On this system, R is available via environment module 'r'. By
-# specifying 'r/4.5.1', 'module load r/4.5.1' will be added to
-# the submitted job script.
-plan(future.batchtools::batchtools_slurm, resources = list(
-  time = "00:10:00", mem = "400M", nodes=1, ntasks=4,
-  modules = c("r/4.5.1")
-))
-
-# Give it a spin
-f <- future({
-  data.frame(
-    hostname = Sys.info()[["nodename"]],
-          os = Sys.info()[["sysname"]],
-       cores = unname(parallelly::availableCores()),
-     modules = Sys.getenv("LOADEDMODULES")
-  )
-})
-info <- value(f)
-print(info)
-#>   hostname    os cores  modules
-#> 1      n12 Linux     4  r/4.5.1
-```
-
-## Demos
-
-The **[future]** package provides a demo using futures for calculating
-a set of Mandelbrot planes.  The demo does not assume anything about
-what type of futures are used.  _The user has full control of how
-futures are evaluated_.  For instance, to use local batchtools
-futures, run the demo as:
-
-```r
-library(future)
-plan(future.batchtools::batchtools_local)
-demo("mandelbrot", package = "future", ask = FALSE)
-```
 
 
 [batchtools]: https://cran.r-project.org/package=batchtools
-[brew]: https://cran.r-project.org/package=brew
 [future]: https://cran.r-project.org/package=future
 [future.batchtools]: https://cran.r-project.org/package=future.batchtools
-[batchtools configuration]: https://batchtools.mlr-org.com/articles/batchtools.html
 [TORQUE]: https://en.wikipedia.org/wiki/TORQUE
 [Slurm]: https://en.wikipedia.org/wiki/Slurm_Workload_Manager
 [Sun/Oracle Grid Engine (SGE)]: https://en.wikipedia.org/wiki/Oracle_Grid_Engine
 [Load Sharing Facility (LSF)]: https://en.wikipedia.org/wiki/Platform_LSF
 [OpenLava]: https://en.wikipedia.org/wiki/OpenLava
-[Docker Swarm]: https://docs.docker.com/swarm/
 
 ## Installation
 R package future.batchtools is available on [CRAN](https://cran.r-project.org/package=future.batchtools) and can be installed in R as: