Skip to content

Commit b92a424

Browse files
committed
Add a content checking example
1 parent 61ea24b commit b92a424

File tree

3 files changed

+102
-26
lines changed

3 files changed

+102
-26
lines changed

docs/side_quests/nf-test.md

Lines changed: 98 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -496,7 +496,7 @@ We don't have to write tests for every part of the pipeline, but the more tests
496496

497497
Let's start with the `sayHello` process.
498498

499-
Let's use the `nf-test generate` command again to generate tests for the entire pipeline.
499+
Let's use the `nf-test generate` command again to generate tests for the process.
500500

501501
```bash
502502
nf-test generate process main.nf
@@ -710,7 +710,7 @@ If, in the course of future development, something in the code changes that caus
710710
- If it turns out that something in the code broke, we will have to fix it, with the expectation that the fixed code will pass the test.
711711
- If it is an expected change (e.g., the tool has been improved and the results are better) then we will need to update the snapshot to accept the new output as the reference to match. nf-test has a parameter `--update-snapshot` for this purpose.
712712

713-
For now though, we can run the test again and see the test should pass:
713+
We can run the test again and see the test should pass:
714714

715715
```console title="nf-test process pass with snapshot"
716716
> nf-test test tests/main.sayhello.nf.test
@@ -730,7 +730,101 @@ SUCCESS: Executed 1 tests in 1.685s
730730

731731
Success! The test passes because the `sayHello` process ran successfully and the output matched the snapshot.
732732

733-
### 2.3. Test the `convertToUpper` process
733+
### 2.3 Alternative to Snapshots: Direct Content Assertions
734+
735+
While snapshots are great for catching any changes in output, sometimes you want to verify specific content without being so strict about the entire file matching. For example:
736+
737+
- When parts of the output might change (timestamps, random IDs, etc.) but certain key content must be present
738+
- When you want to check for specific patterns or values in the output
739+
- When you want to make the test more explicit about what constitutes success
740+
741+
Here's how we could modify our test to check specific content:
742+
743+
**Before:**
744+
745+
```groovy title="tests/main.sayhello.nf.test"
746+
process {
747+
"""
748+
test("Should run without failures and produce correct output") {
749+
750+
when {
751+
params {
752+
// define parameters here. Example:
753+
// outdir = "tests/results"
754+
}
755+
process {
756+
"""
757+
input[0] = "hello"
758+
"""
759+
}
760+
}
761+
762+
then {
763+
assert process.success
764+
assert snapshot(process.out).match()
765+
}
766+
767+
}
768+
769+
"""
770+
}
771+
```
772+
773+
774+
**After:**
775+
776+
```groovy title="tests/main.sayhello.nf.test"
777+
test("Should run without failures and contain expected greeting") {
778+
when {
779+
params {
780+
// define parameters here
781+
}
782+
process {
783+
"""
784+
input[0] = "hello"
785+
"""
786+
}
787+
}
788+
789+
then {
790+
assert process.success
791+
assert path(process.out[0][0]).readLines().contains('hello')
792+
assert !path(process.out[0][0]).readLines().contains('HELLO')
793+
}
794+
}
795+
```
796+
797+
Note that nf-test sees the process outputs as a list of lists, so `process.out[0][0]` is fetching the first part of the first channel item (or 'emission') from this process.
798+
799+
This approach:
800+
- Makes it clear exactly what we expect in the output
801+
- Is more resilient to irrelevant changes in the output
802+
- Provides better error messages when tests fail
803+
- Allows for more complex validations (regex patterns, numerical comparisons, etc.)
804+
805+
Let's run the test to see if it works.
806+
807+
```bash title="nf-test pipeline pass"
808+
nf-test test tests/main.sayhello.nf.test
809+
```
810+
811+
```console title="Process test fails"
812+
> nf-test test tests/main.sayhello.nf.test
813+
814+
🚀 nf-test 0.9.2
815+
https://www.nf-test.com
816+
(c) 2021 - 2024 Lukas Forer and Sebastian Schoenherr
817+
818+
819+
Test Process sayHello
820+
821+
Test [58df4e4b] 'Should run without failures and contain expected greeting' PASSED (7.196s)
822+
823+
824+
SUCCESS: Executed 1 tests in 7.208s
825+
```
826+
827+
### 2.4. Test the `convertToUpper` process
734828

735829
Let's open the `tests/main.converttoupper.nf.test` file and take a look at the contents:
736830

@@ -774,7 +868,7 @@ We now need to supply a single input file to the convertToUpper process, which i
774868
- We could re-use the existing data/greetings.csv file
775869
- We could create it on the fly within the test
776870

777-
For now, let's re-use the existing data/greetings.csv file using the example we used with the pipeline level test. As before, we can name the test to better reflect what we're testing.
871+
For now, let's re-use the existing data/greetings.csv file using the example we used with the pipeline level test. As before, we can name the test to better reflect what we're testing, but this time let's leave it to 'snapshot' the content rather than checking for specific strings (as we did in the other process).
778872

779873
**Before:**
780874

side-quests/solutions/nf-test/tests/main.sayhello.nf.test

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,10 @@ nextflow_process {
44
script "main.nf"
55
process "sayHello"
66

7-
test("Should run without failures") {
8-
7+
test("Should run without failures and contain expected greeting") {
98
when {
109
params {
11-
// define parameters here. Example:
12-
// outdir = "tests/results"
10+
// define parameters here
1311
}
1412
process {
1513
"""
@@ -20,9 +18,9 @@ nextflow_process {
2018

2119
then {
2220
assert process.success
23-
assert snapshot(process.out).match()
21+
assert path(process.out[0][0]).readLines().contains('hello')
22+
assert !path(process.out[0][0]).readLines().contains('HELLO')
2423
}
25-
2624
}
2725

2826
}

side-quests/solutions/nf-test/tests/main.sayhello.nf.test.snap

Lines changed: 0 additions & 16 deletions
This file was deleted.

0 commit comments

Comments
 (0)