Skip to content

pipeline level test with snapshots #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FloWuenne merged 3 commits into from
Feb 27, 2025
Merged

pipeline level test with snapshots #80

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d540c20
pipeline level test with snapshots
maxulysse Feb 27, 2025
f16cc4e
update CHANGELOG
maxulysse Feb 27, 2025
5ba18ac
update nftignore and snapshot
maxulysse Feb 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Thank you to everyone else that has contributed by reporting bugs, enhancements
- [PR #74](https://github.com/seqeralabs/nf-aggregate/pull/74) - Add process for generating Benchmark reports
- [PR #75](https://github.com/seqeralabs/nf-aggregate/pull/75) - Skip failed jobs in benchmarking report
- [PR #78](https://github.com/seqeralabs/nf-aggregate/pull/78) - Sync with nf-core tools 3.2.0
- [PR #90](https://github.com/seqeralabs/nf-aggregate/pull/90) - Snapshots for pipeline level tests with nft-utils

## [[0.5.0](https://github.com/seqeralabs/nf-aggregate/releases/tag/0.5.0)] - 2024-11-12

Expand Down
5 changes: 5 additions & 0 deletions nf-test.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@ config {

// run all test with the defined docker profile from the main nextflow.config
profile "test"

// load the necessary plugins
plugins {
load "nft-utils@0.0.3"
}
}
9 changes: 9 additions & 0 deletions tests/.nftignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.DS_Store
multiqc/multiqc_data/multiqc.log
multiqc/multiqc_data/multiqc_data.json
multiqc/multiqc_data/multiqc_general_stats.txt
multiqc/multiqc_data/multiqc_sources.txt
multiqc/multiqc_plots/{svg,pdf,png}/*.{svg,pdf,png}
multiqc/multiqc_report.html
nf-core_*/gantt/*_gantt.html
pipeline_info/*.{html,json,txt,yml}
29 changes: 21 additions & 8 deletions tests/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,36 @@
nextflow_pipeline {

name "Test Pipeline main.nf"
name "Test pipeline"
script "../main.nf"

tag "pipeline"
tag "pipeline_nf-aggregate"
tag "nf-aggregate"

test("Should run without failures") {
test("-profile test") {

when {
params {
outdir = "results"
outdir = "$outputDir"
}
}

then {
assert workflow.success
// stable_name: All files + folders in ${params.outdir}/ with a stable name
def stable_name = getAllFilesFromDir(params.outdir, relative: true, includeDir: true, ignore: ['pipeline_info/*.{html,json,txt}'])
// stable_path: All files in ${params.outdir}/ with stable content
def stable_path = getAllFilesFromDir(params.outdir, ignoreFile: 'tests/.nftignore')
assertAll(
{ assert workflow.success},
{ assert snapshot(
// Number of successful tasks
workflow.trace.succeeded().size(),
// pipeline versions.yml file for multiqc from which Nextflow version is removed because we tests pipelines on multiple Nextflow versions
removeNextflowVersion("$outputDir/pipeline_info/collated_software_mqc_versions.yml"),
// All stable path name, with a relative path
stable_name,
// All files with stable contents
stable_path
).match() }
)
}

}

}
163 changes: 163 additions & 0 deletions tests/main.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
{
"-profile test": {
"content": [
10,
{
"Workflow": {
"seqeralabs/nf-aggregate": "0.6.0"
},
"PLOT_RUN_GANTT": {
"python": "3.9.19",
"pandas": "1.1.5",
"plotly_express": "0.4.1",
"click": "8.0.1"
},
"SEQERA_RUNS_DUMP": {
"seqera-cli": "0.9.2 (build 39a2ec6)"
}
},
[
"multiqc",
"multiqc/multiqc_data",
"multiqc/multiqc_data/multiqc.log",
"multiqc/multiqc_data/multiqc_citations.txt",
"multiqc/multiqc_data/multiqc_data.json",
"multiqc/multiqc_data/multiqc_general_stats.txt",
"multiqc/multiqc_data/multiqc_seqera_cli.txt",
"multiqc/multiqc_data/multiqc_software_versions.txt",
"multiqc/multiqc_data/multiqc_sources.txt",
"multiqc/multiqc_data/seqera_cli_cost_plot.txt",
"multiqc/multiqc_data/seqera_cli_cpu_time_plot.txt",
"multiqc/multiqc_data/seqera_cli_process_statuses_plot.txt",
"multiqc/multiqc_data/seqera_cli_wall_time_plot.txt",
"multiqc/multiqc_plots",
"multiqc/multiqc_plots/pdf",
"multiqc/multiqc_plots/pdf/general_stats_table.pdf",
"multiqc/multiqc_plots/pdf/seqera_cli_cost_plot.pdf",
"multiqc/multiqc_plots/pdf/seqera_cli_cpu_time_plot.pdf",
"multiqc/multiqc_plots/pdf/seqera_cli_process_statuses_plot-cnt.pdf",
"multiqc/multiqc_plots/pdf/seqera_cli_process_statuses_plot-pct.pdf",
"multiqc/multiqc_plots/pdf/seqera_cli_wall_time_plot.pdf",
"multiqc/multiqc_plots/png",
"multiqc/multiqc_plots/png/general_stats_table.png",
"multiqc/multiqc_plots/png/seqera_cli_cost_plot.png",
"multiqc/multiqc_plots/png/seqera_cli_cpu_time_plot.png",
"multiqc/multiqc_plots/png/seqera_cli_process_statuses_plot-cnt.png",
"multiqc/multiqc_plots/png/seqera_cli_process_statuses_plot-pct.png",
"multiqc/multiqc_plots/png/seqera_cli_wall_time_plot.png",
"multiqc/multiqc_plots/svg",
"multiqc/multiqc_plots/svg/general_stats_table.svg",
"multiqc/multiqc_plots/svg/seqera_cli_cost_plot.svg",
"multiqc/multiqc_plots/svg/seqera_cli_cpu_time_plot.svg",
"multiqc/multiqc_plots/svg/seqera_cli_process_statuses_plot-cnt.svg",
"multiqc/multiqc_plots/svg/seqera_cli_process_statuses_plot-pct.svg",
"multiqc/multiqc_plots/svg/seqera_cli_wall_time_plot.svg",
"multiqc/multiqc_report.html",
"nf-core_ampliseq",
"nf-core_ampliseq/gantt",
"nf-core_ampliseq/gantt/2lXd1j7OwZVfxh_gantt.html",
"nf-core_ampliseq/runs_dump",
"nf-core_ampliseq/runs_dump/2lXd1j7OwZVfxh",
"nf-core_ampliseq/runs_dump/2lXd1j7OwZVfxh/service-info.json",
"nf-core_ampliseq/runs_dump/2lXd1j7OwZVfxh/workflow-launch.json",
"nf-core_ampliseq/runs_dump/2lXd1j7OwZVfxh/workflow-load.json",
"nf-core_ampliseq/runs_dump/2lXd1j7OwZVfxh/workflow-metadata.json",
"nf-core_ampliseq/runs_dump/2lXd1j7OwZVfxh/workflow-metrics.json",
"nf-core_ampliseq/runs_dump/2lXd1j7OwZVfxh/workflow-tasks.json",
"nf-core_ampliseq/runs_dump/2lXd1j7OwZVfxh/workflow.json",
"nf-core_rnaseq",
"nf-core_rnaseq/gantt",
"nf-core_rnaseq/gantt/4Bi5xBK6E2Nbhj_gantt.html",
"nf-core_rnaseq/gantt/4LWT4uaXDaGcDY_gantt.html",
"nf-core_rnaseq/runs_dump",
"nf-core_rnaseq/runs_dump/3iFMo0NtH1Byvy",
"nf-core_rnaseq/runs_dump/3iFMo0NtH1Byvy/service-info.json",
"nf-core_rnaseq/runs_dump/3iFMo0NtH1Byvy/workflow-launch.json",
"nf-core_rnaseq/runs_dump/3iFMo0NtH1Byvy/workflow-load.json",
"nf-core_rnaseq/runs_dump/3iFMo0NtH1Byvy/workflow-metadata.json",
"nf-core_rnaseq/runs_dump/3iFMo0NtH1Byvy/workflow-metrics.json",
"nf-core_rnaseq/runs_dump/3iFMo0NtH1Byvy/workflow-tasks.json",
"nf-core_rnaseq/runs_dump/3iFMo0NtH1Byvy/workflow.json",
"nf-core_rnaseq/runs_dump/4Bi5xBK6E2Nbhj",
"nf-core_rnaseq/runs_dump/4Bi5xBK6E2Nbhj/service-info.json",
"nf-core_rnaseq/runs_dump/4Bi5xBK6E2Nbhj/workflow-launch.json",
"nf-core_rnaseq/runs_dump/4Bi5xBK6E2Nbhj/workflow-load.json",
"nf-core_rnaseq/runs_dump/4Bi5xBK6E2Nbhj/workflow-metadata.json",
"nf-core_rnaseq/runs_dump/4Bi5xBK6E2Nbhj/workflow-metrics.json",
"nf-core_rnaseq/runs_dump/4Bi5xBK6E2Nbhj/workflow-tasks.json",
"nf-core_rnaseq/runs_dump/4Bi5xBK6E2Nbhj/workflow.json",
"nf-core_rnaseq/runs_dump/4LWT4uaXDaGcDY",
"nf-core_rnaseq/runs_dump/4LWT4uaXDaGcDY/service-info.json",
"nf-core_rnaseq/runs_dump/4LWT4uaXDaGcDY/workflow-launch.json",
"nf-core_rnaseq/runs_dump/4LWT4uaXDaGcDY/workflow-load.json",
"nf-core_rnaseq/runs_dump/4LWT4uaXDaGcDY/workflow-metadata.json",
"nf-core_rnaseq/runs_dump/4LWT4uaXDaGcDY/workflow-metrics.json",
"nf-core_rnaseq/runs_dump/4LWT4uaXDaGcDY/workflow-tasks.json",
"nf-core_rnaseq/runs_dump/4LWT4uaXDaGcDY/workflow.json",
"nf-core_scrnaseq",
"nf-core_scrnaseq/gantt",
"nf-core_scrnaseq/gantt/38QXz4OfQDpwOV_gantt.html",
"nf-core_scrnaseq/runs_dump",
"nf-core_scrnaseq/runs_dump/38QXz4OfQDpwOV",
"nf-core_scrnaseq/runs_dump/38QXz4OfQDpwOV/service-info.json",
"nf-core_scrnaseq/runs_dump/38QXz4OfQDpwOV/workflow-launch.json",
"nf-core_scrnaseq/runs_dump/38QXz4OfQDpwOV/workflow-load.json",
"nf-core_scrnaseq/runs_dump/38QXz4OfQDpwOV/workflow-metadata.json",
"nf-core_scrnaseq/runs_dump/38QXz4OfQDpwOV/workflow-metrics.json",
"nf-core_scrnaseq/runs_dump/38QXz4OfQDpwOV/workflow-tasks.json",
"nf-core_scrnaseq/runs_dump/38QXz4OfQDpwOV/workflow.json",
"pipeline_info",
"pipeline_info/collated_software_mqc_versions.yml"
],
[
"multiqc_citations.txt:md5,4c806e63a283ec1b7e78cdae3a923d4f",
"multiqc_seqera_cli.txt:md5,f7dc6872f13a1c9fe989b84540a940ff",
"multiqc_software_versions.txt:md5,ffd82573d0a86a65849475c90973c980",
"seqera_cli_cost_plot.txt:md5,5f6c0eec1ee86691a6d2914e163a5767",
"seqera_cli_cpu_time_plot.txt:md5,390f4051d6092a84a2b3f73ab10643c5",
"seqera_cli_process_statuses_plot.txt:md5,13414743798b07baebe846fb8d20bdde",
"seqera_cli_wall_time_plot.txt:md5,abc859f5fe0dabfcded8c658de70725a",
"service-info.json:md5,5738071394871f50948ae77090d4af50",
"workflow-launch.json:md5,a842489f283947ea92a4f88826bbabf0",
"workflow-load.json:md5,947517e30d5485ad97601923f2ccb2f5",
"workflow-metadata.json:md5,99ec56750319402d9eed979b51527d0e",
"workflow-metrics.json:md5,a862f20aabadbc5ab0be91642dffc00c",
"workflow-tasks.json:md5,7301efc1db5ee06cd57be6b6ea901f36",
"workflow.json:md5,59f3848c26e6b99155240a61e57222bf",
"service-info.json:md5,5738071394871f50948ae77090d4af50",
"workflow-launch.json:md5,6fdbe8c3a57a9e0e11d65d0057a074e4",
"workflow-load.json:md5,7fe53d3eb1bd3e8ab882d011a751b832",
"workflow-metadata.json:md5,01f59714a52a57456219a95cdb29cb5a",
"workflow-metrics.json:md5,15b8927b842c6fd1fc738af2e8d3462e",
"workflow-tasks.json:md5,26adadf43dd7f5d9996b3ff6c5698ff6",
"workflow.json:md5,0f7da8cd7d2bfe2b72bc0dc4cbb0fbc0",
"service-info.json:md5,5738071394871f50948ae77090d4af50",
"workflow-launch.json:md5,032225ac0735972950537a3b5e913662",
"workflow-load.json:md5,4f02d5a24ab89aa648cd4346785c8f2c",
"workflow-metadata.json:md5,4347508e26b93645bd393157856f0860",
"workflow-metrics.json:md5,13a5b5d7447fad4a8baa053d1abf85e5",
"workflow-tasks.json:md5,577a7472816b7729012a9291d97ff150",
"workflow.json:md5,658701f8d7af1b7112534a7f58cd9aa7",
"service-info.json:md5,5738071394871f50948ae77090d4af50",
"workflow-launch.json:md5,1ccf29020c410b00fdc02537ce7a813e",
"workflow-load.json:md5,2f5abdb8efc1fdb249378cb1af093a4a",
"workflow-metadata.json:md5,2fc6fdbe5bfc7cc90b92725c586c3cd3",
"workflow-metrics.json:md5,5cb0d8e859991831eff666eed1c97b3e",
"workflow-tasks.json:md5,bf390ebbe51b2dc7aea4c60a4a02ee97",
"workflow.json:md5,a81b1319f6a6038d1b2de9f8c7ddb8b6",
"service-info.json:md5,5738071394871f50948ae77090d4af50",
"workflow-launch.json:md5,56995097a449167c11769a566c7d2716",
"workflow-load.json:md5,bebdd4917d3b769a656744ba25241898",
"workflow-metadata.json:md5,3dcba41db4802a3a43e6660e7cd9b759",
"workflow-metrics.json:md5,fa417298bc5c9c73c1c2bc1daf6e0406",
"workflow-tasks.json:md5,84b7baa01cb692faa5466d8b577f61d6",
"workflow.json:md5,8e1bcf993c75642a766adea2d3b4ee30"
]
],
"meta": {
"nf-test": "0.9.2",
"nextflow": "25.01.0"
},
"timestamp": "2025-02-27T13:21:17.488661076"
}
}
11 changes: 6 additions & 5 deletions tests/nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
/*
========================================================================================
Nextflow config file for running nf-test tests
========================================================================================
*/

params {
// Base directory for nf-core/modules test data
modules_testdata_base_path = 's3://ngi-igenomes/testdata/nf-core/modules/'
}

timeline.enabled = false
report.enabled = false
trace.enabled = false
dag.enabled = false