Merge pull request #1461 from nf-core/swf_rnaseq_prepro_lint

Add FASTQ linting during preprocessing

Author: pinin4fjords

.github/workflows/ci.yml

@@ -68,6 +68,11 @@ jobs:
       - name: Check out pipeline code
         uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
 
+      - uses: actions/setup-java@8df1039502a15bceb9433410b1a100fbe190c53b # v4
+        with:
+          distribution: "temurin"
+          java-version: "17"
+
       - name: Set up Nextflow
         uses: nf-core/setup-nextflow@v2
         with:

CHANGELOG.md

@@ -3,6 +3,18 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+# 3.18.0dev - xxxx-xx-xx
+
+### Credits
+
+Special thanks to the following for their contributions to the release:
+
+- [Caitlin Winkler](https://github.com/oligomyeggo)
+
+### Enhancements & fixes
+
+- [PR #1461](https://github.com/nf-core/rnaseq/pull/1461) - Add FASTQ linting during preprocessing
+
 ## [[3.17.0](https://github.com/nf-core/rnaseq/releases/tag/3.17.0)] - 2024-10-23
 
 ### Credits

docs/output.md

@@ -19,6 +19,7 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
   - [Pipeline overview](#pipeline-overview)
   - [Preprocessing](#preprocessing)
     - [cat](#cat)
+      [fq lint](#fq-lint)
     - [FastQC](#fastqc)
     - [UMI-tools extract](#umi-tools-extract)
     - [TrimGalore](#trimgalore)
@@ -73,6 +74,20 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
 
 If multiple libraries/runs have been provided for the same sample in the input samplesheet (e.g. to increase sequencing depth) then these will be merged at the very beginning of the pipeline in order to have consistent sample naming throughout the pipeline. Please refer to the [usage documentation](https://nf-co.re/rnaseq/usage#samplesheet-input) to see how to specify these samples in the input samplesheet.
 
+# fq lint
+
+<details markdown="1">
+<summary>Output files</summary>
+
+- `fq_lint/*`
+  - `*.fq_lint.txt`: Linting report per library from `fq lint`.
+
+> **NB:** You will see subdirectories here based on the stage of preprocessing for the files that have been linted, for example `raw`, `trimmed`.
+
+</details>
+
+[fq lint](https://github.com/stjude-rust-labs/fq) runs several checks on input FASTQ files. It will fail with a non-zero error code when issues are found, which will terminate the workflow execution. In the absence of this, the successful linting produces the logs you will find here.
+
 ### FastQC
 
 <details markdown="1">

docs/usage.md

@@ -27,6 +27,12 @@ CONTROL_REP1,AEG588A1_S1_L003_R1_001.fastq.gz,AEG588A1_S1_L003_R2_001.fastq.gz,a
 CONTROL_REP1,AEG588A1_S1_L004_R1_001.fastq.gz,AEG588A1_S1_L004_R2_001.fastq.gz,auto
 ```
 
+### Linting
+
+By default, the pipeline will run [fq lint](https://github.com/stjude-rust-labs/fq) on all input FASTQ files, both at the start of preprocessing and after each preprocessing step that manipulates FASTQ files. If errors are found, and error will be reported and the workflow will stop.
+
+The `extra_fqlint_args` parameter can be manipulated to disable [any validator](https://github.com/stjude-rust-labs/fq?tab=readme-ov-file#validators) from `fq` you wish. For example, we have found that checks on the names of paired reads are prone to failure, so that check is disabled by default (setting `extra_fqlint_args` to `--disable-validator P001`).
+
 ### Strandedness Prediction
 
 If you set the strandedness value to `auto`, the pipeline will sub-sample the input FastQ files to 1 million reads, use Salmon Quant to automatically infer the strandedness, and then propagate this information through the rest of the pipeline. This behavior is controlled by the `--stranded_threshold` and `--unstranded_threshold` parameters, which are set to 0.8 and 0.1 by default, respectively. This means:

modules.json

@@ -57,6 +57,11 @@
                         "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["fastq_fastqc_umitools_fastp", "fastq_fastqc_umitools_trimgalore"]
                     },
+                    "fq/lint": {
+                        "branch": "master",
+                        "git_sha": "2c0260ed80daeca9c6dfa477a4daf04ff336dc37",
+                        "installed_by": ["fastq_qc_trim_filter_setstrandedness", "modules"]
+                    },
                     "fq/subsample": {
                         "branch": "master",
                         "git_sha": "a1abf90966a2a4016d3c3e41e228bfcbd4811ccc",
@@ -341,7 +346,7 @@
                     },
                     "fastq_qc_trim_filter_setstrandedness": {
                         "branch": "master",
-                        "git_sha": "9082d6440bdffbb4f5d9bd9d753361933b3febcb",
+                        "git_sha": "2c0260ed80daeca9c6dfa477a4daf04ff336dc37",
                         "installed_by": ["subworkflows"]
                     },
                     "fastq_subsample_fq_salmon": {