Merge branch 'master' into devcontainers-cleanup

Author: ewels

docs/hello_nextflow/01_hello_world.md

@@ -138,7 +138,7 @@ process sayHello {
 }
 ```
 
-This a very minimal process definition that just contains an `output` definition and the `script` to execute.
+This is a very minimal process definition that just contains an `output` definition and the `script` to execute.
 
 The `output` definition includes the `path` qualifier, which tells Nextflow this should be handled as a path (includes both directory paths and files).
 Another common qualifier is `val`.
@@ -172,7 +172,7 @@ workflow {
 }
 ```
 
-This a very minimal **workflow** definition.
+This is a very minimal **workflow** definition.
 In a real-world pipeline, the workflow typically contains multiple calls to **processes** connected by **channels**, and the processes expect one or more variable **input(s)**.
 
 You'll learn how to add variable inputs later in this training module; and you'll learn how to add more processes and connect them by channels in Part 3 of this course.
@@ -479,7 +479,7 @@ In the process block, make the following code change:
 
 _Before:_
 
-```groovy title="hello-channels.nf" linenums="6"
+```groovy title="hello-world.nf" linenums="6"
 process sayHello {
 
     publishDir 'results', mode: 'copy'
@@ -490,7 +490,7 @@ process sayHello {
 
 _After:_
 
-```groovy title="hello-channels.nf" linenums="6"
+```groovy title="hello-world.nf" linenums="6"
 process sayHello {
 
     publishDir 'results', mode: 'copy'

docs/hello_nextflow/02_hello_channels.md

@@ -522,7 +522,7 @@ Here we added the operator on the next line for readability, but you can add ope
 
 #### 3.2.2. Add `view()` to inspect channel contents
 
-We could run this right away to test if it works, but while we're at it, we're also going to add a couple of [`view()`](https://www.nextflow.io/docs/latest/reference/operator.html#view) directives, which allow us to inspect the contents of a channel.
+We could run this right away to test if it works, but while we're at it, we're also going to add a couple of [`view()`](https://www.nextflow.io/docs/latest/reference/operator.html#view) operators, which allow us to inspect the contents of a channel.
 You can think of `view()` as a debugging tool, like a `print()` statement in Python, or its equivalent in other languages.
 
 In the workflow block, make the following code change:
@@ -540,12 +540,25 @@ _After:_
 ```groovy title="hello-channels.nf" linenums="31"
     // create a channel for inputs
     greeting_ch = Channel.of(greetings_array)
-                         .view { "Before flatten: $it" }
+                         .view { greeting -> "Before flatten: $greeting" }
                          .flatten()
-                         .view { "After flatten: $it" }
+                         .view { greeting -> "After flatten: $greeting" }
 ```
 
-Here `$it` is an implicit variable that represents each individual item loaded in a channel.
+We are using an operator _closure_ here - the curly brackets.
+This code executes for each item in the channel.
+We define a temporary variable for the inner value, here called `greeting` (it could be anything).
+This variable is only used within the scope of that closure.
+
+In this example, `$greeting` represents each individual item loaded in a channel.
+
+!!! note "Note on `$it`"
+
+    In some pipelines you may see a special variable called `$it` used inside operator closures.
+    This is an _implicit_ variable that allows a short-hand access to the inner variable,
+    without needing to define it with a `->`.
+
+    We prefer to be explicit to aid code clarity, as such the `$it` syntax is discouraged and will slowly be phased out of the Nextflow language.
 
 #### 3.2.3. Run the workflow
 
@@ -723,9 +736,9 @@ _After:_
 ```groovy title="hello-channels.nf" linenums="31"
 // create a channel for inputs from a CSV file
 greeting_ch = Channel.fromPath(params.greeting)
-                     .view { "Before splitCsv: $it" }
+                     .view { csv -> "Before splitCsv: $csv" }
                      .splitCsv()
-                     .view { "After splitCsv: $it" }
+                     .view { csv -> "After splitCsv: $csv" }
 ```
 
 As you can see, we also include before/after view statements while we're at it.
@@ -787,7 +800,7 @@ This is what the syntax looks like:
 
 This means 'for each element in the channel, take the first of any items it contains'.
 
-So let's apply that to our CVS parsing.
+So let's apply that to our CSV parsing.
 
 #### 4.3.1. Apply `map()` to the channel
 
@@ -798,21 +811,21 @@ _Before:_
 ```groovy title="hello-channels.nf" linenums="31"
 // create a channel for inputs from a CSV file
 greeting_ch = Channel.fromPath(params.greeting)
-                     .view { "Before splitCsv: $it" }
+                     .view { csv -> "Before splitCsv: $csv" }
                      .splitCsv()
-                     .view { "After splitCsv: $it" }
+                     .view { csv -> "After splitCsv: $csv" }
 ```
 
 _After:_
 
 ```groovy title="hello-channels.nf" linenums="31"
 // create a channel for inputs from a CSV file
 greeting_ch = Channel.fromPath(params.greeting)
-                     .view { "Before splitCsv: $it" }
+                     .view { csv -> "Before splitCsv: $csv" }
                      .splitCsv()
-                     .view { "After splitCsv: $it" }
+                     .view { csv -> "After splitCsv: $csv" }
                      .map { item -> item[0] }
-                     .view { "After map: $it" }
+                     .view { csv -> "After map: $csv" }
 ```
 
 Once again we include another `view()` call to confirm that the operator does what we expect.

docs/hello_nextflow/03_hello_workflow.md

@@ -485,8 +485,8 @@ _After:_
     collectGreetings(convertToUpper.out.collect())
 
     // optional view statements
-    convertToUpper.out.view { "Before collect: $it" }
-    convertToUpper.out.collect().view { "After collect: $it" }
+    convertToUpper.out.view { greeting -> "Before collect: $greeting" }
+    convertToUpper.out.collect().view { greeting -> "After collect: $greeting" }
 }
 ```
 
@@ -816,11 +816,9 @@ _After:_
     collectGreetings(convertToUpper.out.collect(), params.batch)
 
     // emit a message about the size of the batch
-    collectGreetings.out.count.view { "There were $it greetings in this batch" }
+    collectGreetings.out.count.view { num_greetings -> "There were $num_greetings greetings in this batch" }
 ```
 
-Here we are using `$it` in the same way we did earlier, as an implicit variable to access the contents of the channel.
-
 !!! note
 
     There are a few other ways we could achieve a similar result, including some more elegant ones like the `count()` operator, but this allows us to show how to handle multiple outputs, which is what we care about.

docs/hello_nextflow/05_hello_containers.md

@@ -189,7 +189,7 @@ You can see that the filesystem inside the container is different from the files
     When you run a container, it is isolated from the host system by default.
     This means that the container can't access any files on the host system unless you explicitly allow it to do so.
 
-You will learn how to do that in a minute.
+    You will learn how to do that in a minute.
 
 #### 1.3.2. Run the desired tool command(s)
 
@@ -434,7 +434,7 @@ _Before:_
     collectGreetings(convertToUpper.out.collect(), params.batch)
 
     // emit a message about the size of the batch
-    collectGreetings.out.count.view{ "There were $it greetings in this batch" }
+    collectGreetings.out.count.view{ num_greetings -> "There were $num_greetings greetings in this batch" }
 ```
 
 _After:_
@@ -444,7 +444,7 @@ _After:_
     collectGreetings(convertToUpper.out.collect(), params.batch)
 
     // emit a message about the size of the batch
-    collectGreetings.out.count.view{ "There were $it greetings in this batch" }
+    collectGreetings.out.count.view{ num_greetings -> "There were $num_greetings greetings in this batch" }
 
     // generate ASCII art of the greetings with cowpy
     cowpy(collectGreetings.out.outfile, params.character)

docs/hello_nextflow/index.md

@@ -12,7 +12,7 @@ The rise of big data has made it increasingly necessary to be able to analyze an
 
 During this training, you will be introduced to Nextflow in a series of complementary hands-on workshops.
 
-Let's get started! Click on the "Open in GitHub Codespaces" button below.
+Let's get started! Click on the "Open in GitHub Codespaces" button below to launch the training environment (preferably in a separate tab), then read on while it loads.
 
 [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/nextflow-io/training?quickstart=1&ref=master)
 

docs/nf4_science/genomics/index.md

@@ -11,10 +11,9 @@ It builds on the [Hello Nextflow](../../hello_nextflow/) beginner training and d
 
 Specifically, this course demonstrates how to implement a simple variant calling pipeline with [GATK](https://gatk.broadinstitute.org/) (Genome Analysis Toolkit), a widely used software package for analyzing high-throughput sequencing data.
 
-!!! note
+Let's get started! Click on the "Open in GitHub Codespaces" button below to launch the training environment (preferably in a separate tab), then read on while it loads.
 
-    Don't worry if you're not familiar with GATK specifically.
-    We'll summarize the necessary concepts as we go, and the workflow implementation principles we demonstrate here apply broadly to any command line tool that processes genomics data.
+[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/nextflow-io/training?quickstart=1&ref=master)
 
 ## Learning objectives
 
@@ -40,9 +39,3 @@ The course assumes some minimal familiarity with the following:
 - Foundational Nextflow concepts and tooling covered in the [Hello Nextflow](../../hello_nextflow/) beginner training.
 
 For technical requirements and environment setup, see the [Environment Setup](../../envsetup/) mini-course.
-
-## Get started
-
-To get started, open the training environment by clicking the 'Open in GitHub Codespaces' button below.
-
-[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/nextflow-io/training?quickstart=1&ref=master)

docs/nf4_science/rnaseq/00_orientation.md

@@ -0,0 +1,93 @@
+# Orientation
+
+The training environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself.
+However, you do need a (free) account to log in, and you should take a few minutes to familiarize yourself with the interface.
+
+If you have not yet done so, please the [Environment Setup](../../envsetup/) mini-course before going any further.
+
+## Materials provided
+
+Throughout this training course, we'll be working in the `nf4-science/rnaseq/` directory, which you need to move into when you open the training workspace.
+This directory contains all the code files, test data and accessory files you will need.
+
+Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the training workspace in the VSCode interface.
+Alternatively, you can use the `tree` command.
+Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity.
+
+Here we generate a table of contents to the second level down:
+
+```bash
+tree . -L 3
+```
+
+If you run this inside `nf4-science/rnaseq`, you should see the following output:
+
+```console title="Directory contents"
+.
+├── data
+│   ├── genome.fa
+│   ├── paired-end.csv
+│   ├── reads
+│   │   ├── ENCSR000COQ1_1.fastq.gz
+│   │   ├── ENCSR000COQ1_2.fastq.gz
+│   │   ├── ENCSR000COQ2_1.fastq.gz
+│   │   ├── ENCSR000COQ2_2.fastq.gz
+│   │   ├── ENCSR000COR1_1.fastq.gz
+│   │   ├── ENCSR000COR1_2.fastq.gz
+│   │   ├── ENCSR000COR2_1.fastq.gz
+│   │   ├── ENCSR000COR2_2.fastq.gz
+│   │   ├── ENCSR000CPO1_1.fastq.gz
+│   │   ├── ENCSR000CPO1_2.fastq.gz
+│   │   ├── ENCSR000CPO2_1.fastq.gz
+│   │   └── ENCSR000CPO2_2.fastq.gz
+│   └── single-end.csv
+├── nextflow.config
+├── rnaseq.nf
+└── solutions
+    ├── modules
+    │   ├── fastqc.nf
+    │   ├── fastqc_pe.nf
+    │   ├── hisat2_align.nf
+    │   ├── hisat2_align_pe.nf
+    │   ├── multiqc.nf
+    │   ├── trim_galore.nf
+    │   └── trim_galore_pe.nf
+    ├── rnaseq-2.1.nf
+    ├── rnaseq-2.2.nf
+    ├── rnaseq-2.3.nf
+    ├── rnaseq-3.1.nf
+    ├── rnaseq-3.2.nf
+    └── rnaseq_pe-3.3.nf
+
+```
+
+!!!note
+
+    Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course.
+    This is just meant to give you an overview.
+
+**Here's a summary of what you should know to get started:**
+
+- **The `rnaseq.nf` file** is the outline if the workflow script we will work to develop.
+
+- **The file `nextflow.config`** is a configuration file that sets minimal environment properties. You can ignore it for now.
+
+- **The `data` directory** contains input data and related resources:
+
+  - _A reference genome_ called `genome.fa` consisting of a small region of the human chromosome 20 (from hg19/b37).
+  - _RNAseq data_ that has been subset to a small region to keep the file sizes down, in the `reads/` directory.
+  - _CSV files_ listing the IDs and paths of the example data files, for processing in batches.
+
+- **The `solutions` directory** contains the completed workflow scripts and modules that result from each step of the course.
+  They are intended to be used as a reference to check your work and troubleshoot any issues.
+  The number in the filename corresponds to the step of the relevant part of the course.
+
+!!!tip
+
+    If for whatever reason you move out of this directory, you can always run this command to return to it:
+
+    ```bash
+    cd /workspaces/training/nf4-science/rnaseq
+    ```
+
+Now, to begin the course, click on the arrow in the bottom right corner of this page.