release 1.0.0 (#16)

* update description

* test verified commit

* test signed commit

* adjust samplesheet

* include bcftools for pass filtering

* added bcftools_split

Signed-off-by: SusiJo <susanne.jodoin@gmx.de>

* first draft of querynator module

* querynator module built and tested

* started to incorporate querynator in pipeline

* first draft of incorporated querynator/query_cgi

* cgi query running

* cancer type fixed

* made unzipping possible for cgi

* cgi + first draft civic

* all done, first draft

* Update nextflow.config

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

* Update bin/check_samplesheet.py

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

* next try

* Update bin/check_samplesheet.py

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

* Update bin/check_samplesheet.py

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

* Update modules/local/querynator/cgiapi.nf

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

* Update workflows/variantmtb.nf

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

* Modules

* some changes

* querynator update

* querynator updated

* docs added

* docs added

* remove fastqc

* changelog

* beautify code

* remove all multiqc

* minor changes

* minimal test included

* cancer types

* add paths

* description

* disable tabix

* Update README.md

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

* Update docs/usage.md

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

* moved CGI credentials to nf secrets

* prettier

* added log files in general

* removed CGI credentials from params

* ECLint

* Revert "removed CGI credentials from params"

This reverts commit fafeff7.

keep info in schema

* updated nf-core/bcftools to v1.18

* removed CGI credentials from input channel

* Revert "removed CGI credentials from input channel"

This reverts commit e086a16.

* added meta2 argument, as required since bcftools1.18

* fixed broken link

* added reference genome meta info to bcftools input channel

* potential fix #4

* Revert "potential fix #4"

This reverts commit bac4cbe.

* another potential fix #4

* yet another potential fix #4

* removed fasta default value, so it can be reassigned by script and genome flag

* added qbicpipelines header and logo

* Template update for nf-core/tools version 2.13.1

* added qbic logo and minor fixes

* nf-core lint

* prettier

* lint

* prettier

* try ignoring utils_nfcore_pipeline

* removed obsolete cgi options and grouped other options

* Revert "try ignoring utils_nfcore_pipeline"

This reverts commit 5402ddf.

* cleanup & description of cgi credentials

* removed some todos

* lint

* cleanup

* minor nf-test fix

* undo qbicpipelines logo to pass linting

* undo qbicpipelines logo

* fixed typo

* fix linting issue by ignoring images

* minor fix

* added/updated tests

* minor fix

* added citations

* moved fasta channel init to main workflow

* added link to list of cancer types at querynator repo

* added BGZF compressed vcf to full test

* refactored

* prettier

* added enum to genome

* fixed igenomes and linting

* update querynator to 0.4.2

* added name to contributors

* removed unused script

* minor adjustments

* Revert "minor adjustments"

This reverts commit 4b4c3eb.

* r1.0.0 (#15)

* feat!: specify cancer type for cgi & civic in samplesheet (#5)

* refactored meta info parsing

* bumped querynator to 0.5.1
moved cancer type from params to samplesheet

* updated usage

* updated usage

* lint

* update bcftools/norm to 1.18

* update bcftools/view to 1.18

* updated gunzip module

* updated custom/dumpsoftwareversions module

* updated utils_nfcore_pipeline subworkflow

* lint

* added cancer type defaults

* updated module gunzip

* updated module tabix/tabix

* updated module tabix/bgziptabix

* bump querynator to 0.5.4

* docs: update CHANGELOG

* fix: disable parallelization for QUERYCGI

CGI blocks multiple parallel requests to their API

fixes #10

* build: bump version to 1.0.0

* test: run multiple samples for the full test

* Bump querynator 0.5.5

* chore: remove obsolete TODOs

* chore: update nf-core modules

* chore: update nf-core subworkflows

* chore(querynator): update to 0.5.5

* fix(docs): fix github actions badges

* feat: add civic evidence filters

* chore: bump querynator to 0.6.0

* chore: update nf-core modules

* chore: update nf-core subworkflows

* docs: update changelog

* docs: add actionability profile

---------

Signed-off-by: SusiJo <susanne.jodoin@gmx.de>
Co-authored-by: SusiJo <susanne.jodoin@gmx.de>
Co-authored-by: mapo9 <mark.polster@uni-tuebingen.de>
Co-authored-by: Mark Polster <58596435+mapo9@users.noreply.github.com>
Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>
Co-authored-by: Niclas Grote <zxmgc83@thanos.am10.uni-tubingen.de>

Author: 4 people

.devcontainer/devcontainer.json

@@ -0,0 +1,20 @@
+{
+    "name": "nfcore",
+    "image": "nfcore/gitpod:latest",
+    "remoteUser": "gitpod",
+    "runArgs": ["--privileged"],
+
+    // Configure tool-specific properties.
+    "customizations": {
+        // Configure properties specific to VS Code.
+        "vscode": {
+            // Set *default* container specific settings.json values on container create.
+            "settings": {
+                "python.defaultInterpreterPath": "/opt/conda/bin/python"
+            },
+
+            // Add the IDs of extensions you want installed when the container is created.
+            "extensions": ["ms-python.python", "ms-python.vscode-pylance", "nf-core.nf-core-extensionpack"]
+        }
+    }
+}

.editorconfig

@@ -18,7 +18,20 @@ end_of_line = unset
 insert_final_newline = unset
 trim_trailing_whitespace = unset
 indent_style = unset
-indent_size = unset
+[/subworkflows/nf-core/**]
+charset = unset
+end_of_line = unset
+insert_final_newline = unset
+trim_trailing_whitespace = unset
+indent_style = unset
 
 [/assets/email*]
 indent_size = unset
+
+# ignore Readme
+[README.md]
+indent_style = unset
+
+# ignore python
+[*.{py,md}]
+indent_style = unset

.gitattributes

@@ -1,3 +1,4 @@
 *.config linguist-language=nextflow
+*.nf.test linguist-language=nextflow
 modules/nf-core/** linguist-generated
 subworkflows/nf-core/** linguist-generated

.github/CONTRIBUTING.md

@@ -9,6 +9,7 @@ Please use the pre-filled template to save time.
 However, don't be put off by this template - other more general issues and suggestions are welcome!
 Contributions to the code are even more welcome ;)
 
+> [!NOTE]
 > If you need help using or modifying nf-core/variantmtb then the best place to ask is on the nf-core Slack [#variantmtb](https://nfcore.slack.com/channels/variantmtb) channel ([join our Slack here](https://nf-co.re/join/slack)).
 
 ## Contribution workflow
@@ -25,6 +26,12 @@ If you're not used to this workflow with git, you can start with some [docs from
 
 ## Tests
 
+You have the option to test your changes locally by running the pipeline. For receiving warnings about process selectors and other `debug` information, it is recommended to use the debug profile. Execute all the tests with the following command:
+
+```bash
+nf-test test --profile debug,test,docker --verbose
+```
+
 When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
 Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.
 
@@ -85,7 +92,7 @@ Once there, use `nf-core schema build` to add to `nextflow_schema.json`.
 
 Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
 
-The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
+The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.
 
 ### Naming schemes
 
@@ -101,3 +108,18 @@ If you are using a new feature from core Nextflow, you may bump the minimum requ
 ### Images and figures
 
 For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/developers/design_guidelines).
+
+## GitHub Codespaces
+
+This repo includes a devcontainer configuration which will create a GitHub Codespaces for Nextflow development! This is an online developer environment that runs in your browser, complete with VSCode and a terminal.
+
+To get started:
+
+- Open the repo in [Codespaces](https://github.com/nf-core/variantmtb/codespaces)
+- Tools installed
+  - nf-core
+  - Nextflow
+
+Devcontainer specs:
+
+- [DevContainer config](.devcontainer/devcontainer.json)

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -42,9 +42,9 @@ body:
     attributes:
       label: System information
       description: |
-        * Nextflow version _(eg. 21.10.3)_
+        * Nextflow version _(eg. 23.04.0)_
         * Hardware _(eg. HPC, Desktop, Cloud)_
         * Executor _(eg. slurm, local, awsbatch)_
-        * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
+        * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
         * OS _(eg. CentOS Linux, macOS, Linux Mint)_
         * Version of nf-core/variantmtb _(eg. 1.1, 1.5, 1.8.2)_

.github/PULL_REQUEST_TEMPLATE.md

@@ -15,10 +15,11 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/vari
 
 - [ ] This comment contains a description of changes (with reason).
 - [ ] If you've fixed a bug or added code that should be tested, add tests!
-  - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/variantmtb/tree/master/.github/CONTRIBUTING.md)
-  - [ ] If necessary, also make a PR on the nf-core/variantmtb _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
+- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/variantmtb/tree/master/.github/CONTRIBUTING.md)
+- [ ] If necessary, also make a PR on the nf-core/variantmtb _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
 - [ ] Make sure your code lints (`nf-core lint`).
 - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
+- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
 - [ ] Usage Documentation in `docs/usage.md` is updated.
 - [ ] Output Documentation in `docs/output.md` is updated.
 - [ ] `CHANGELOG.md` is updated.

.github/workflows/awsfulltest.yml

@@ -14,17 +14,26 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Launch workflow via tower
-        uses: nf-core/tower-action@v3
-        # TODO nf-core: You can customise AWS full pipeline tests as required
+        uses: seqeralabs/action-tower-launch@v2
+        # nf-core: You can customise AWS full pipeline tests as required
         # Add full size test data (but still relatively small datasets for few samples)
         # on the `test_full.config` test runs with only one set of parameters
         with:
           workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
           access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
           compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
+          revision: ${{ github.sha }}
           workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/variantmtb/work-${{ github.sha }}
           parameters: |
             {
+              "hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
               "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/variantmtb/results-${{ github.sha }}"
             }
-          profiles: test_full,aws_tower
+          profiles: test_full
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: Tower debug log file
+          path: |
+            tower_action_*.log
+            tower_action_*.json

.github/workflows/awstest.yml

@@ -12,14 +12,22 @@ jobs:
     steps:
       # Launch workflow using Tower CLI tool action
       - name: Launch workflow via tower
-        uses: nf-core/tower-action@v3
+        uses: seqeralabs/action-tower-launch@v2
         with:
           workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
           access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
           compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
+          revision: ${{ github.sha }}
           workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/variantmtb/work-${{ github.sha }}
           parameters: |
             {
               "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/variantmtb/results-test-${{ github.sha }}"
             }
-          profiles: test,aws_tower
+          profiles: test
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: Tower debug log file
+          path: |
+            tower_action_*.log
+            tower_action_*.json

.github/workflows/branch.yml

@@ -13,13 +13,13 @@ jobs:
       - name: Check PRs
         if: github.repository == 'nf-core/variantmtb'
         run: |
-          { [[ ${{github.event.pull_request.head.repo.full_name }} == nf-core/variantmtb ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
+          { [[ ${{github.event.pull_request.head.repo.full_name }} == nf-core/variantmtb ]] && [[ $GITHUB_HEAD_REF == "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
 
       # If the above check failed, post a comment on the PR explaining the failure
       # NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
       - name: Post PR comment
         if: failure()
-        uses: mshick/add-pr-comment@v1
+        uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
         with:
           message: |
             ## This PR is against the `master` branch :x:

.github/workflows/ci.yml

@@ -10,7 +10,10 @@ on:
 
 env:
   NXF_ANSI_LOG: false
-  CAPSULE_LOG: none
+
+concurrency:
+  group: "${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}"
+  cancel-in-progress: true
 
 jobs:
   test:
@@ -20,30 +23,23 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        # Nextflow versions
-        include:
-          # Test pipeline minimum Nextflow version
-          - NXF_VER: "21.10.3"
-            NXF_EDGE: ""
-          # Test latest edge release of Nextflow
-          - NXF_VER: ""
-            NXF_EDGE: "1"
+        NXF_VER:
+          - "23.04.0"
+          - "latest-everything"
     steps:
       - name: Check out pipeline code
-        uses: actions/checkout@v2
+        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
 
       - name: Install Nextflow
-        env:
-          NXF_VER: ${{ matrix.NXF_VER }}
-          # Uncomment only if the edge release is more recent than the latest stable release
-          # See https://github.com/nextflow-io/nextflow/issues/2467
-          # NXF_EDGE: ${{ matrix.NXF_EDGE }}
-        run: |
-          wget -qO- get.nextflow.io | bash
-          sudo mv nextflow /usr/local/bin/
+        uses: nf-core/setup-nextflow@v1
+        with:
+          version: "${{ matrix.NXF_VER }}"
+
+      - name: Disk space cleanup
+        uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
 
       - name: Run pipeline with test data
-        # TODO nf-core: You can customise CI pipeline run tests as required
+        # nf-core: You can customise CI pipeline run tests as required
         # For example: adding multiple test runs with different parameters
         # Remember that you can parallelise this by using strategy.matrix
         run: |