Merge pull request #173 from nextflow-io/2.5.0dev

Release 2.5.0

Author: nvnieuwk

.github/workflows/build.yml

@@ -51,4 +51,4 @@ jobs:
         with:
           name: test-reports-jdk-${{ matrix.java_version }}
           path: |
-            ./plugins/nf-schema/build/reports/tests/test/**
+            ./plugins/nf-schema/build/reports/tests/test/**

.github/workflows/examples.yml

@@ -0,0 +1,36 @@
+name: Check examples
+on:
+  push:
+    tags-ignore:
+      - '*'
+  pull_request:
+jobs:
+  check_examples:
+    name: Check examples output
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 1
+          submodules: true
+
+      - name: Setup Java 21
+        uses: actions/setup-java@v3
+        with:
+          java-version: 21
+          distribution: 'adopt'
+          architecture: x64
+          cache: gradle
+
+      - name: Setup Nextflow
+        uses: nf-core/setup-nextflow@v2
+        with:
+          version: "latest-stable"
+
+      - name: Install nf-schema
+        run: make install
+
+      - name: Check examples
+        run: bash examples/check_examples.sh

CHANGELOG.md

@@ -1,5 +1,46 @@
 # nextflow-io/nf-schema: Changelog
 
+# Version 2.5.0
+
+## New features
+
+1. Added a new configuration option: `validation.help.enumLength` which sets the maximum length of enum values in the help message. The default is set to the value in the `COLUMNS` environment variable or 100 character if that variable isn't set.
+2. Added top-level schema description for detailed help `--help <args>`. If `<args>` is associated with a schema, then the top-level fields will be shown similarly to `--help` for the main schema (#146).
+3. Added a Troubleshooting and FAQ page to the documentation.
+
+## Changes
+
+1. The plugin now properly validates cloud storage files instead of skipping them. Exotic errors will be added to the error messages instead of failing the validation outright.
+2. Migrated to the new observer class in Nextflow 25.04.0
+3. Rework the deprecated `paramsHelp()` function to allow pipeline developers a fully fledged alternative to the help messages created via the configuration options. This change enables the use of dynamic help message with the strict configuration syntax introduced in Nextflow 25.04.0.
+4. Updated the error message when the JSON schema file is invalid to include the full file name.
+5. The ANSI log setting of Nextflow is now used to determine whether or not the log should be monochrome. This setting will take priority over the `validation.monochromeLogs` configuration option.
+6. Updated the examples in the [examples](examples/) directory and added automatic checking for validity of these examples.
+7. Refactored logic for parsing the `jsonschema` validation result into a new `ValidationResult` class.
+
+## Bug fixes
+
+1. CSV and TSV files with trailing commas or tabs will now be properly sanitized. This fixes issues with CSV and TSV files that contained empty header columns.
+2. Unidentified parameters are no longer printed out on failure of the parameter validation. This is to prevent a bug where all parameters would be printed out on failure.
+3. Fixed an issue where default values of `null` weren't being set correctly in `samplesheetToList()`.
+4. Fixed an undocumented limit of `3MB` for `YAML` format samplesheets (new limit is `50MB`).
+5. Fixed issue where an empty string in a yaml for a file type string format would throw a Java error instead of reporting a proper validation error.
+
+## Logging configuration
+
+This update contains a rework of the logging configuration. The `valdiation.logging` configuration scope has been added with options to replace and expand the current logging configuration options. These options can all take the following values:
+
+- `skip`: Skip logging
+- `debug`: Log debug messages (only printed in the `.nextflow.log` file)
+- `info`: Log info messages (also printed in the terminal)
+- `warn`: Log warning messages (also printed in the terminal, but in yellow)
+- `error`: Fail the pipeline and print the error message
+
+| Old option                           | New option                               | Description                                                 |
+| ------------------------------------ | ---------------------------------------- | ----------------------------------------------------------- |
+| `validation.failUnrecognisedParams`  | `validation.logging.unrecognisedParams`  | The logging level for unrecognised parameters.              |
+| `validation.failUnrecognisedHeaders` | `validation.logging.unrecognisedHeaders` | The logging level for unrecognised headers in samplesheets. |
+
 # Version 2.4.2
 
 ## Bug fixes

README.md

@@ -25,7 +25,7 @@ Declare the plugin in your Nextflow pipeline configuration file:
 
 ```groovy title="nextflow.config"
 plugins {
-  id 'nf-schema@2.4.1'
+  id 'nf-schema@2.5.0'
 }
 ```
 

build.gradle

@@ -1,5 +1,5 @@
 plugins {
-    id 'io.nextflow.nextflow-plugin' version '0.0.1-alpha2'
+    id 'io.nextflow.nextflow-plugin' version '0.0.1-alpha3'
 }
 
 dependencies {
@@ -8,14 +8,15 @@ dependencies {
     implementation 'com.sanctionco.jmail:jmail:1.6.3' // Needed for e-mail format validation
 }
 
-version = '2.4.2'
+version = '2.5.0'
 
 nextflowPlugin {
-    nextflowVersion = '24.10.0'
+    nextflowVersion = '25.04.0'
 
     provider = 'nextflow'
     className = 'nextflow.validation.ValidationPlugin'
     extensionPoints = [
+        'nextflow.validation.config.ValidationConfig',
         'nextflow.validation.ValidationExtension',
         'nextflow.validation.ValidationObserverFactory'
     ]

buildSrc/build.gradle

@@ -32,7 +32,7 @@ This option can either be a path relative to the root of the pipeline directory
 
 ## monochromeLogs
 
-This option can be used to turn of the colored logs from nf-validation. This can be useful if you run a Nextflow pipeline in an environment that doesn't support colored logging.
+This option can be used to turn of the colored logs from nf-schema. This can be useful if you run a Nextflow pipeline in an environment that doesn't support colored logging. This option will automatically be set to `true` when ANSI logging is disabled in Nextflow.
 
 ```groovy
 validation.monochromeLogs = <true|false> // default: false
@@ -46,22 +46,6 @@ This option can be used to make the type validation more lenient. In normal case
 validation.lenientMode = <true|false> // default: false
 ```
 
-## failUnrecognisedParams
-
-By default the `validateParameters()` function will only give a warning if an unrecognised parameter has been given. This usually indicates that a typo has been made and can be easily overlooked when the plugin only emits a warning. You can turn this warning into an error with the `failUnrecognisedParams` option.
-
-```groovy
-validation.failUnrecognisedParams = <true|false> // default: false
-```
-
-## failUnrecognisedHeaders
-
-By default the `samplesheetToList()` function will only give a warning if an unrecognised header is present in the samplesheet. This usually indicates that a typo has been made and can be easily overlooked when the plugin only emits a warning. You can turn this warning into an error with the `failUnrecognisedHeaders` option.
-
-```groovy
-validation.failUnrecognisedHeaders = <true|false> // default: false
-```
-
 ## showHiddenParams
 
 !!! deprecated
@@ -213,6 +197,16 @@ validation.help.afterText = "Please cite the pipeline owners when using this pip
 
     All color values (like `\033[0;31m`, which means the color red) will be filtered out when `validation.monochromeLogs` is set to `true`
 
+### enumLength
+
+This option can be used to set the maximum length of enum values in the help message. This is useful when the enum values are very long and would make the help message hard to read.
+
+The default value is set to the value in the `COLUMNS` environment variable or 100 characters if that variable isn't set. Set this option to `-1` to disable the length limit.
+
+```groovy
+validation.help.enumLength = <integer>
+```
+
 ## Summary
 
 The `validation.summary` config scope can be used to configure the output of the `paramsSummaryLog()` function.
@@ -250,3 +244,32 @@ Takes a list of parameter names to exclude from the parameters summary created b
 ```groovy
 validation.summary.hideParams = ["param1", "nested.param"] // default: []
 ```
+
+## Logging
+
+The `validation.logging` config scope can be used to configure the logging of the plugin.
+These config options can receive the following values:
+
+- `skip`: Skip logging
+- `debug`: Log debug messages (only printed in the `.nextflow.log` file)
+- `info`: Log info messages (also printed in the terminal)
+- `warn`: Log warning messages (also printed in the terminal, but in yellow)
+- `error`: Fail the pipeline and print the error message
+
+This scope contains the following options:
+
+### unrecognisedParams
+
+This option can be used to configure the logging of unrecognised parameters.
+
+```groovy
+validation.logging.unrecognisedParams = "warn" // default: "warn"
+```
+
+### unrecognisedHeaders
+
+This option can be used to configure the logging of unrecognised headers in samplesheets.
+
+```groovy
+validation.logging.unrecognisedHeaders = "warn" // default: "warn"
+```