Skip to content

Commit 740a9be

Browse files
Extended validation to cover two more fields in the CSV (#704)
* Extended validation to cover two more fields in the CSV This is to match an upcoming change in the Hello Nextflow greetings file, and provides opportunity to see other types of validation rules in action * Minor fixes Line numbers, improve the clarity of creating the invalid test case, + fixes #705 * Invalide input case works with updated greetings.csv * Minor fixes * line highlights killing me * fix sentence * Minor fixes to the index page, orientation adn part 1 of hello nf-core * Minor improvements to part 2 of hello nf-core * Minor fix to part 3 of nf-core hello * Address Marcel's comment * fix url to wow side quest * One more SQ url fix --------- Co-authored-by: Marcel Ribeiro-Dantas <marcel@seqera.io>
1 parent 1c4ff25 commit 740a9be

File tree

18 files changed

+199
-107
lines changed

18 files changed

+199
-107
lines changed

docs/hello_nf-core/00_orientation.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,9 +75,10 @@ If you run this inside `hello-nf-core`, you should see the following output.
7575
├── core-hello-part2
7676
├── core-hello-part3
7777
├── core-hello-part4
78+
├── core-hello-part5
7879
└── core-hello-start
7980

80-
8 directories, 3 files
81+
9 directories, 3 files
8182
```
8283

8384
!!! note

docs/hello_nf-core/01_run_demo.md

Lines changed: 19 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ Let's start by locating the nf-core/demo pipeline on the project website at [nf-
1414

1515
### 1.1. Find the pipeline on the website
1616

17-
In your web browser, go to https://nf-co.re/pipelines/ and type `demo` in the search bar.
17+
In your web browser, go to [https://nf-co.re/pipelines/](https://nf-co.re/pipelines/) and type `demo` in the search bar.
1818

1919
![search results](./img/search-results.png)
2020

@@ -105,6 +105,8 @@ tree -L 2 $NXF_HOME/assets/
105105
/workspaces/.nextflow/assets/
106106
└── nf-core
107107
└── demo
108+
109+
2 directories, 0 files
108110
```
109111

110112
!!! note
@@ -130,6 +132,8 @@ tree -L 2 pipelines
130132
pipelines
131133
└── nf-core
132134
└── demo
135+
136+
2 directories, 0 files
133137
```
134138

135139
Now we can more easily peek into the source code as needed.
@@ -297,7 +301,7 @@ Here's the console output from the pipeline:
297301

298302
If your output matches that, congratulations! You've just run your first nf-core pipeline.
299303

300-
You'll notice that there is more a lot more console output than when you run a basic Nextflow pipeline.
304+
You'll notice that there is a lot more console output than when you run a basic Nextflow pipeline.
301305
There's a header that includes a summary of the pipeline's version, inputs and outputs, and a few elements of configuration.
302306

303307
!!! note
@@ -500,21 +504,19 @@ tree -L 3 pipelines/nf-core/demo/modules
500504
pipelines/nf-core/demo/modules
501505
└── nf-core
502506
├── fastqc
503-
   ├── environment.yml
504-
   ├── main.nf
505-
   ├── meta.yml
506-
   └── tests
507+
├── environment.yml
508+
├── main.nf
509+
├── meta.yml
510+
└── tests
507511
├── multiqc
508-
   ├── environment.yml
509-
   ├── main.nf
510-
   ├── meta.yml
511-
   └── tests
512+
├── environment.yml
513+
├── main.nf
514+
├── meta.yml
515+
└── tests
512516
└── seqtk
513517
└── trim
514-
├── environment.yml
515-
├── main.nf
516-
├── meta.yml
517-
└── tests
518+
519+
7 directories, 6 files
518520
```
519521

520522
Here you see that the `fastqc` and `multiqc` modules sit at the top level within the `nf-core` modules, whereas the `trim` module sits under the toolkit that it belongs to, `seqtk`.
@@ -553,6 +555,8 @@ tree -L 3 pipelines/nf-core/demo/subworkflows
553555
├── main.nf
554556
├── meta.yml
555557
└── tests
558+
559+
9 directories, 7 files
556560
```
557561

558562
As noted above, the `nf-core/demo` pipeline does not include any analysis-specific subworkflows, so all the subworkflows we see here are so-called 'housekeeping' or 'utility' workflows, as denoted by the `utils_` prefix in their names.
@@ -626,4 +630,4 @@ Take a break! That was a lot. When you're feeling refreshed and ready, move on t
626630

627631
!!! tip
628632

629-
If you would like to learn how to compose workflows with subworkflows before moving on to the next part, check out the [Workflows of Workflows](../side_quests/workflows_of_workflows/) Side Quest.
633+
If you would like to learn how to compose workflows with subworkflows before moving on to the next part, check out the [Workflows of Workflows](../side_quests/workflows_of_workflows.md) Side Quest.

docs/hello_nf-core/02_rewrite_hello.md

Lines changed: 42 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -214,6 +214,8 @@ tree core-hello-results
214214
├── hello_software_versions.yml
215215
├── params_2025-11-21_04-47-18.json
216216
└── pipeline_dag_2025-11-21_04-47-18.html
217+
218+
1 directory, 6 files
217219
```
218220

219221
You can take a peek at the reports to see what was run, and the answer is: nothing at all!
@@ -275,7 +277,7 @@ workflow HELLO {
275277
*/
276278
```
277279

278-
Compared to a basic Nextflow workflow like the one developed in Hello Nextflow, you'll notice a few things that are new here (highlighted lines above):
280+
Compared to a basic Nextflow workflow like the one developed in [Hello Nextflow](../hello_nextflow/index.md), you'll notice a few things that are new here (highlighted lines above):
279281

280282
- The workflow block has a name
281283
- Workflow inputs are declared using the `take:` keyword and the channel construction is moved up to the parent workflow
@@ -286,7 +288,7 @@ These are optional features of Nextflow that make the workflow **composable**, m
286288

287289
!!! note "Composable workflows in depth"
288290

289-
The [Workflows of Workflows](../side_quests/workflows_of_workflows/) Side Quest explores workflow composition in much greater depth, including how to compose multiple workflows together and manage complex data flows between them. We're introducing composability here because it's a fundamental requirement of the nf-core template architecture, which uses nested workflows to organize pipeline initialization, the main analysis workflow, and completion tasks into separate, reusable components.
291+
The [Workflows of Workflows](../side_quests/workflows_of_workflows.md) Side Quest explores workflow composition in much greater depth, including how to compose multiple workflows together and manage complex data flows between them. We're introducing composability here because it's a fundamental requirement of the nf-core template architecture, which uses nested workflows to organize pipeline initialization, the main analysis workflow, and completion tasks into separate, reusable components.
290292

291293
We are going to need to plug the relevant logic from our workflow of interest into that structure.
292294
The first step for that is to make our original workflow composable.
@@ -371,6 +373,8 @@ tree original-hello/
371373
│ ├── cowpy.nf
372374
│ └── sayHello.nf
373375
└── nextflow.config
376+
377+
1 directory, 6 files
374378
```
375379

376380
Feel free to run it to satisfy yourself that it works:
@@ -478,15 +482,15 @@ Now, replace the channel construction with a simple `take` statement declaring e
478482

479483
This leaves the details of how the inputs are provided up to the parent workflow.
480484

481-
While we're at it, we can also comment out the line
485+
While we're at it, we can also comment out the line `params.greeting = 'greetings.csv'`
482486

483487
=== "After"
484488

485489
```groovy title="original-hello/hello.nf" linenums="3" hl_lines="4"
486490
/*
487491
* Pipeline parameters
488492
*/
489-
params.greeting = 'greetings.csv'
493+
//params.greeting = 'greetings.csv'
490494
params.batch = 'test-batch'
491495
params.character = 'turkey'
492496
```
@@ -497,13 +501,11 @@ While we're at it, we can also comment out the line
497501
/*
498502
* Pipeline parameters
499503
*/
500-
// params.greeting = 'greetings.csv'
504+
params.greeting = 'greetings.csv'
501505
params.batch = 'test-batch'
502506
params.character = 'turkey'
503507
```
504508

505-
params.greeting = 'greetings.csv'
506-
507509
!!! note
508510

509511
If you have the Nextflow language server extension installed, the syntax checker will light up your code with red squiggles.
@@ -566,7 +568,7 @@ This is a net new addition to the code compared to the original workflow.
566568

567569
If you've done all the changes as described, your workflow should now look like this:
568570

569-
```groovy title="original-hello/hello.nf" linenums="1" hl_lines="15 17-19 21 37-38"
571+
```groovy title="original-hello/hello.nf" linenums="1" hl_lines="16 18-20 22 36-37"
570572
#!/usr/bin/env nextflow
571573
572574
/*
@@ -802,6 +804,8 @@ tree core-hello/modules
802804
├── convertToUpper.nf
803805
├── cowpy.nf
804806
└── sayHello.nf
807+
808+
1 directory, 4 files
805809
```
806810

807811
Now let's set up the module import statements.
@@ -1301,6 +1305,32 @@ Key points:
13011305
- **Absolute paths**: By using `${projectDir}`, we create an absolute path, which is important for test data that ships with the pipeline.
13021306
- **Test data location**: nf-core pipelines typically store test data in the `assets/` directory within the pipeline repository for small test files, or reference external test datasets for larger files.
13031307

1308+
And while we're at it, let's tighten the default resource limits to ensure this will run on very basic machines (like the minimal VMs in Github Codespaces):
1309+
1310+
=== "After"
1311+
1312+
```groovy title="core-hello/config/test.config" linenums="13" hl_lines="3-4"
1313+
process {
1314+
resourceLimits = [
1315+
cpus: 2,
1316+
memory: '4.GB',
1317+
time: '1.h'
1318+
]
1319+
}
1320+
```
1321+
1322+
=== "Before"
1323+
1324+
```groovy title="core-hello/config/test.config" linenums="13" hl_lines="3-4"
1325+
process {
1326+
resourceLimits = [
1327+
cpus: 4,
1328+
memory: '15.GB',
1329+
time: '1.h'
1330+
]
1331+
}
1332+
```
1333+
13041334
This completes the code modifications we need to do.
13051335

13061336
### 4.4. Run the pipeline with the test profile
@@ -1382,6 +1412,8 @@ tree core-hello-results
13821412
├── params_2025-11-21_07-29-41.json
13831413
└── pipeline_dag_2025-11-21_04-47-18.html
13841414
└── pipeline_dag_2025-11-21_07-29-37.html
1415+
1416+
1 directory, 12 files
13851417
```
13861418

13871419
You see we got another set of execution reports in addition to the ones we got from the first run, when the workflow was still just a placeholder.
@@ -1416,6 +1448,8 @@ tree results
14161448
├── UPPER-Bonjour-output.txt
14171449
├── UPPER-Hello-output.txt
14181450
└── UPPER-Holà-output.txt
1451+
1452+
0 directories, 10 files
14191453
```
14201454

14211455
Ah, there they are, mixed in with the outputs of earlier runs of the original Hello pipeline.

docs/hello_nf-core/03_use_module.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ nf-core modules list remote | grep 'cat/cat'
7575
```
7676

7777
```console title="Output"
78-
cat/cat
78+
cat/cat
7979
```
8080

8181
Just keep in mind the that `grep` approach will only pull out results with the search term in their name, which would not work for `cat_cat`.
@@ -156,7 +156,8 @@ cd core-hello
156156
nf-core modules install cat/cat
157157
```
158158

159-
The tool will first prompt you to specify a repository type.
159+
The tool may first prompt you to specify a repository type.
160+
(If not, skip down to "Finally, the tool will proceed to install the module.")
160161

161162
??? example "Output"
162163

@@ -177,7 +178,7 @@ The tool will first prompt you to specify a repository type.
177178
Modules repository
178179
```
179180

180-
Press enter to accept the default response (`Pipeline`) and continue.
181+
If so, press enter to accept the default response (`Pipeline`) and continue.
181182

182183
The tool will then offer to amend the configuration of your project to avoid this prompt in the future.
183184

@@ -675,7 +676,7 @@ Since `cowpy` doesn't accept metadata tuples yet (we'll fix this in the next par
675676

676677
=== "After"
677678

678-
```groovy title="core-hello/workflows/hello.nf" linenums="26" hl_lines="16-17"
679+
```groovy title="core-hello/workflows/hello.nf" linenums="26" hl_lines="16-17 20"
679680
// emit a greeting
680681
sayHello(ch_samplesheet)
681682

@@ -700,7 +701,7 @@ Since `cowpy` doesn't accept metadata tuples yet (we'll fix this in the next par
700701

701702
=== "Before"
702703

703-
```groovy title="core-hello/workflows/hello.nf" linenums="26"
704+
```groovy title="core-hello/workflows/hello.nf" linenums="26" hl_lines="17"
704705
// emit a greeting
705706
sayHello(ch_samplesheet)
706707

@@ -722,7 +723,7 @@ Since `cowpy` doesn't accept metadata tuples yet (we'll fix this in the next par
722723

723724
The `.map{ meta, file -> file }` operation extracts the file from the `[metadata, file]` tuple produced by `CAT_CAT` into a new channel, `ch_for_cowpy`.
724725

725-
Then it's just a matter of passing `ch_for_cowpy` to `cowpy` instead of `collectGreetings.out.outfile`.
726+
Then it's just a matter of passing `ch_for_cowpy` to `cowpy` instead of `collectGreetings.out.outfile` in that last line.
726727

727728
!!! note
728729

docs/hello_nf-core/04_make_module.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -736,7 +736,7 @@ To override the default `publishDir` directive, you can simply add your own dire
736736

737737
For example, you could override the default for a single process using the `withName:` selector, as in this example where we add a custom `publishDir` directive for the 'cowpy' process.
738738

739-
```groovy title="core-hello/conf/modules.config" linenums="13" hl_lines="6-8"
739+
```groovy title="core-hello/conf/modules.config" linenums="13" hl_lines="8-10"
740740
process {
741741
publishDir = [
742742
path: { "${params.outdir}/${task.process.tokenize(':')[-1].tokenize('_')[0].toLowerCase()}" },
@@ -833,7 +833,7 @@ Each file serves a specific purpose:
833833

834834
!!! tip "Learn more about testing"
835835

836-
The generated test file uses nf-test, a testing framework for Nextflow pipelines and modules. To learn how to write and run these tests, see the [nf-test side quest](../../side_quests/nf_test/).
836+
The generated test file uses nf-test, a testing framework for Nextflow pipelines and modules. To learn how to write and run these tests, see the [nf-test side quest](../side_quests/nf_test.md).
837837

838838
The generated `main.nf` includes all the patterns you just learned, plus some additional features:
839839

0 commit comments

Comments
 (0)