From de34cb518e81f0359e994fac326549e566c756bc Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 16 Oct 2025 16:29:14 +0000 Subject: [PATCH 1/5] Initial plan From fcf3ab9b4e0cefd7d3e575b5f69b3d58271ff6bd Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 16 Oct 2025 16:36:52 +0000 Subject: [PATCH 2/5] Add missing version utility functions to plugin API Co-authored-by: edmundmiller <20095261+edmundmiller@users.noreply.github.com> --- .../nfcore/plugin/NfUtilsExtension.groovy | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/main/groovy/nfcore/plugin/NfUtilsExtension.groovy b/src/main/groovy/nfcore/plugin/NfUtilsExtension.groovy index 73f5991..e0fef50 100644 --- a/src/main/groovy/nfcore/plugin/NfUtilsExtension.groovy +++ b/src/main/groovy/nfcore/plugin/NfUtilsExtension.groovy @@ -260,6 +260,48 @@ class NfUtilsExtension extends PluginExtensionPoint { return NfcoreVersionUtils.workflowVersionToChannel(this.session) } + /** + * Parse and flatten YAML string of software versions + * Example: "tool:foo: 1.0.0\nbar: 2.0.0" -> "foo: 1.0.0\nbar: 2.0.0" + * + * @param yamlFile The YAML content as string + * @return Processed YAML string with flattened keys + */ + @Function + String processVersionsFromYAML(String yamlFile) { + return NfcoreVersionUtils.processVersionsFromYAML(yamlFile) + } + + /** + * Get workflow version for pipeline as YAML string + * Returns workflow name, version, and Nextflow version in YAML format + * + * @return YAML string with workflow version information + */ + @Function + String workflowVersionToYAML() { + return NfcoreVersionUtils.workflowVersionToYAML(this.session) + } + + /** + * Combine a list of YAML version strings into a single YAML string + * This is the main function used in nf-core pipelines to collect all software versions. + * Deduplicates entries and appends workflow version info. + * + * Usage pattern in pipelines: + * ch_versions = Channel.empty() + * ch_versions = ch_versions.mix(PROCESS.out.versions) + * softwareVersionsToYAML(ch_versions.unique().collect()) + * .collectFile(name: 'software_versions.yml') + * + * @param chVersions List of YAML version strings from processes + * @return Combined YAML string with all versions and workflow info + */ + @Function + String softwareVersionsToYAML(List chVersions) { + return NfcoreVersionUtils.softwareVersionsToYAML(chVersions, this.session) + } + // --- Citation Management Functions --- /** * Generate citation for a tool from meta.yml at the module level From 19c88ec167df6ce429c9e960d2f22a34a369d4bf Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 16 Oct 2025 16:40:41 +0000 Subject: [PATCH 3/5] Add documentation for new version utility functions Co-authored-by: edmundmiller <20095261+edmundmiller@users.noreply.github.com> --- docs/utilities/NfcoreVersionUtils.md | 209 +++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) diff --git a/docs/utilities/NfcoreVersionUtils.md b/docs/utilities/NfcoreVersionUtils.md index e54bf34..b5fa8db 100644 --- a/docs/utilities/NfcoreVersionUtils.md +++ b/docs/utilities/NfcoreVersionUtils.md @@ -450,6 +450,215 @@ workflow { --- +### `processVersionsFromYAML(String yamlFile)` + +**Description:** +Parses a YAML string of software versions and flattens nested keys. This is a core utility function used internally by `softwareVersionsToYAML` and can be used directly for processing individual version YAML files. + +**Function Signature:** + +```nextflow +String processVersionsFromYAML(String yamlFile) +``` + +**Parameters:** + +- `yamlFile` (String): YAML content as a string to process + +**Returns:** + +- `String`: Processed YAML string with flattened keys + +**Usage Example:** + +```nextflow +include { processVersionsFromYAML } from 'plugin/nf-core-utils' + +// Process a single version YAML +def yamlContent = ''' +tool:fastqc: 0.12.1 +bar: 2.0.0 +''' + +def processed = processVersionsFromYAML(yamlContent) +// Result: "fastqc: 0.12.1\nbar: 2.0.0" +``` + +**Key Behavior:** + +- Removes nested tool prefixes (e.g., `tool:fastqc:` becomes `fastqc:`) +- Preserves version information +- Returns empty string for invalid YAML + +--- + +### `workflowVersionToYAML()` + +**Description:** +Returns workflow version information as a formatted YAML string, including workflow name, version, and Nextflow version. This is the YAML equivalent of `workflowVersionToChannel()`. + +**Function Signature:** + +```nextflow +String workflowVersionToYAML() +``` + +**Parameters:** + +- None (uses session context automatically) + +**Returns:** + +- `String`: YAML-formatted workflow version information + +**Usage Example:** + +```nextflow +include { workflowVersionToYAML } from 'plugin/nf-core-utils' + +workflow { + // Get workflow version as YAML + def workflowYaml = workflowVersionToYAML() + log.info "Workflow info:\n${workflowYaml}" +} +``` + +**Output Example:** + +```yaml +Workflow: + nf-core/rnaseq: v3.12.0-gabcdef1 + Nextflow: 23.04.1 +``` + +**Integration with Version Collection:** + +```nextflow +// Include workflow version in version collection +workflow.onComplete { + def workflowInfo = workflowVersionToYAML() + def versionsFile = file("${params.outdir}/pipeline_info/versions.yml") + versionsFile.text = workflowInfo +} +``` + +--- + +### `softwareVersionsToYAML(List chVersions)` + +**Description:** +Combines a list of YAML version strings into a single unified YAML string. This is the primary function used in nf-core pipelines to collect all software versions from processes. It automatically deduplicates entries and appends workflow version information. + +**Function Signature:** + +```nextflow +String softwareVersionsToYAML(List chVersions) +``` + +**Parameters:** + +- `chVersions` (List): List of YAML version strings collected from processes + +**Returns:** + +- `String`: Combined YAML string with all versions and workflow metadata + +**Usage Example:** + +```nextflow +include { softwareVersionsToYAML } from 'plugin/nf-core-utils' + +workflow { + ch_versions = Channel.empty() + + // Collect versions from processes + FASTQC(samples) + ch_versions = ch_versions.mix(FASTQC.out.versions.first()) + + MULTIQC(reports) + ch_versions = ch_versions.mix(MULTIQC.out.versions.first()) + + // Combine all versions + softwareVersionsToYAML(ch_versions.collect()) + .collectFile( + name: 'software_versions.yml', + storeDir: "${params.outdir}/pipeline_info" + ) +} +``` + +**Standard nf-core Pattern:** + +```nextflow +// This is the standard pattern used in nf-core pipelines +include { softwareVersionsToYAML } from 'plugin/nf-core-utils' + +workflow PIPELINE { + take: + samplesheet + + main: + ch_versions = Channel.empty() + + // Run processes and collect versions + PROCESS_A(samplesheet) + ch_versions = ch_versions.mix(PROCESS_A.out.versions.first()) + + PROCESS_B(PROCESS_A.out.results) + ch_versions = ch_versions.mix(PROCESS_B.out.versions.first()) + + // Generate combined versions file + softwareVersionsToYAML(ch_versions) + .collectFile( + storeDir: "${params.outdir}/pipeline_info", + name: 'nf_core_pipeline_software_mqc_versions.yml', + sort: true, + newLine: true + ).set { ch_collated_versions } + + emit: + versions = ch_versions + collated_versions = ch_collated_versions +} +``` + +**Advanced Usage with Unique Filtering:** + +```nextflow +// Process versions with unique filtering for better performance +workflow { + ch_versions = Channel.empty() + + // Collect all versions + ch_versions = ch_versions + .mix(PROCESS_A.out.versions) + .mix(PROCESS_B.out.versions) + .mix(PROCESS_C.out.versions) + + // Generate unified YAML with deduplication + ch_versions + .unique() // Remove duplicate channel entries + .map { it.text } // Extract YAML content from files + .collect() // Collect all into a list + .map { versionList -> + softwareVersionsToYAML(versionList) + } + .collectFile( + name: 'versions.yml', + storeDir: "${params.outdir}/pipeline_info" + ) +} +``` + +**Key Features:** + +- **Automatic Deduplication**: Removes duplicate version entries +- **Workflow Integration**: Automatically includes workflow and Nextflow versions +- **MultiQC Compatible**: Output format is directly compatible with MultiQC +- **Flexible Input**: Accepts versions from any number of processes + +--- + ## Topic Channel Migration Guide ### Stage 1: Legacy File-Based Approach From d70457edeee6913b3c4971098ff49303934c9a21 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 16 Oct 2025 16:42:05 +0000 Subject: [PATCH 4/5] Update CHANGELOG for new version functions Co-authored-by: edmundmiller <20095261+edmundmiller@users.noreply.github.com> --- CHANGELOG.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 60e8db9..2270941 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] + +### Added + +- Exposed `processVersionsFromYAML()` function for parsing and flattening YAML version strings +- Exposed `workflowVersionToYAML()` function for getting workflow version as YAML string +- Exposed `softwareVersionsToYAML()` function for combining version YAMLs from channels +- Comprehensive documentation for all three new version utility functions + ## [0.3.1] - 2025-09-12 ### Changed From 2b6bd2cf85aabb3f0fd6b4a6a9e869096d1e4312 Mon Sep 17 00:00:00 2001 From: maxulysse Date: Fri, 24 Oct 2025 17:59:29 +0200 Subject: [PATCH 5/5] pre-commit --- docs/utilities/NfcoreVersionUtils.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/utilities/NfcoreVersionUtils.md b/docs/utilities/NfcoreVersionUtils.md index b5fa8db..171e61b 100644 --- a/docs/utilities/NfcoreVersionUtils.md +++ b/docs/utilities/NfcoreVersionUtils.md @@ -527,8 +527,8 @@ workflow { ```yaml Workflow: - nf-core/rnaseq: v3.12.0-gabcdef1 - Nextflow: 23.04.1 + nf-core/rnaseq: v3.12.0-gabcdef1 + Nextflow: 23.04.1 ``` **Integration with Version Collection:**