Skip to content

Commit 57e27cf

Browse files
ewelsewels
authored
Merge pull request nf-core#1782 from nf-core/dev
Dev -> Master for v2.5
2 parents 171127b + 04a699a commit 57e27cf

File tree

167 files changed

+11207
-3743
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

167 files changed

+11207
-3743
lines changed

‎.editorconfig‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,5 @@ trim_trailing_whitespace = true
88
indent_size = 4
99
indent_style = space
1010

11-
[*.{md,yml,yaml,html,css,scss,js}]
11+
[*.{md,yml,yaml,html,css,scss,js,cff}]
1212
indent_size = 2

‎.github/CONTRIBUTING.md‎

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,9 @@ Then install your local fork of nf-core/tools:
3333
pip install -e .
3434
```
3535

36-
## Code formatting with Black
36+
## Code formatting
37+
38+
### Black
3739

3840
All Python code in nf-core/tools must be passed through the [Black Python code formatter](https://black.readthedocs.io/en/stable/).
3941
This ensures a harmonised code formatting style throughout the package, from all contributors.
@@ -51,6 +53,42 @@ You can also set it up to run when you [make a commit](https://black.readthedocs
5153
There is an automated CI check that runs when you open a pull-request to nf-core/tools that will fail if
5254
any code does not adhere to Black formatting.
5355

56+
### isort
57+
58+
All Python code must also be passed through [isort](https://pycqa.github.io/isort/index.html).
59+
This ensures a harmonised imports throughout the package, from all contributors.
60+
61+
To run isort on the command line recursively on the whole repository you can use:
62+
63+
```bash
64+
isort .
65+
```
66+
67+
isort also has [plugins for most common editors](https://github.com/pycqa/isort/wiki/isort-Plugins)
68+
to automatically format code when you hit save.
69+
Or [version control integration](https://pycqa.github.io/isort/docs/configuration/pre-commit.html) to set it up to run when you make a commit.
70+
71+
There is an automated CI check that runs when you open a pull-request to nf-core/tools that will fail if
72+
any code does not adhere to isort formatting.
73+
74+
### pre-commit hooks
75+
76+
This repository comes with [pre-commit](https://pre-commit.com/) hooks for black, isort and Prettier. pre-commit automatically runs checks before a commit is committed into the git history. If all checks pass, the commit is made, if files are changed by the pre-commit hooks, the user is informed and has to stage the changes and attempt the commit again.
77+
78+
You can use the pre-commit hooks if you like, but you don't have to. The CI on Github will run the same checks as the tools installed with pre-commit. If the pre-commit checks pass, then the same checks in the CI will pass, too.
79+
80+
You can install the pre-commit hooks into the development environment by running the following command in the root directory of the repository.
81+
82+
```bash
83+
pre-commit install --install-hooks
84+
```
85+
86+
You can also run all pre-commit hooks without making a commit:
87+
88+
```bash
89+
pre-commit run --all
90+
```
91+
5492
## API Documentation
5593

5694
We aim to write function docstrings according to the [Google Python style-guide](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings). These are used to automatically generate package documentation on the nf-core website using Sphinx.

‎.github/workflows/create-lint-wf.yml‎

Lines changed: 30 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,13 @@
11
name: Create a pipeline and run nf-core linting
22
on: [push, pull_request]
33

4+
# Cancel if a newer run is started
5+
concurrency:
6+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
7+
cancel-in-progress: true
8+
49
env:
510
NXF_ANSI_LOG: false
6-
CAPSULE_LOG: none
711

812
jobs:
913
MakeTestWorkflow:
@@ -12,14 +16,9 @@ jobs:
1216
NXF_ANSI_LOG: false
1317
strategy:
1418
matrix:
15-
# Nextflow versions
16-
include:
17-
# Test pipeline minimum Nextflow version
18-
- NXF_VER: "21.10.3"
19-
NXF_EDGE: ""
20-
# Test latest edge release of Nextflow
21-
- NXF_VER: ""
22-
NXF_EDGE: "1"
19+
NXF_VER:
20+
- "21.10.3"
21+
- "latest-everything"
2322
steps:
2423
# Get the repo code
2524
- uses: actions/checkout@v2
@@ -38,14 +37,9 @@ jobs:
3837
3938
# Set up Nextflow
4039
- name: Install Nextflow
41-
env:
42-
NXF_VER: ${{ matrix.NXF_VER }}
43-
# Uncomment only if the edge release is more recent than the latest stable release
44-
# See https://github.com/nextflow-io/nextflow/issues/2467
45-
# NXF_EDGE: ${{ matrix.NXF_EDGE }}
46-
run: |
47-
wget -qO- get.nextflow.io | bash
48-
sudo mv nextflow /usr/local/bin/
40+
uses: nf-core/setup-nextflow@v1
41+
with:
42+
version: ${{ matrix.NXF_VER }}
4943

5044
# Install the Prettier linting tools
5145
- uses: actions/setup-node@v2
@@ -59,7 +53,7 @@ jobs:
5953

6054
# Build a pipeline from the template
6155
- name: nf-core create
62-
run: nf-core --log-file log.txt create -n testpipeline -d "This pipeline is for testing" -a "Testing McTestface"
56+
run: nf-core --log-file log.txt create -n testpipeline -d "This pipeline is for testing" -a "Testing McTestface" --plain
6357

6458
# Try syncing it before we change anything
6559
- name: nf-core sync
@@ -80,9 +74,13 @@ jobs:
8074
- name: nf-core modules update
8175
run: nf-core --log-file log.txt modules update --dir nf-core-testpipeline --all --no-preview
8276

77+
# Remove TODO statements
78+
- name: remove TODO
79+
run: find nf-core-testpipeline -type f -exec sed -i '/TODO nf-core:/d' {} \;
80+
8381
# Run nf-core linting
8482
- name: nf-core lint
85-
run: nf-core --log-file log.txt lint --dir nf-core-testpipeline --fail-ignored
83+
run: nf-core --log-file log.txt lint --dir nf-core-testpipeline --fail-ignored --fail-warned
8684

8785
# Run the other nf-core commands
8886
- name: nf-core list
@@ -98,11 +96,23 @@ jobs:
9896
run: nf-core --log-file log.txt bump-version --dir nf-core-testpipeline/ 1.1
9997

10098
- name: nf-core lint in release mode
101-
run: nf-core --log-file log.txt lint --dir nf-core-testpipeline --fail-ignored --release
99+
run: nf-core --log-file log.txt lint --dir nf-core-testpipeline --fail-ignored --fail-warned --release
102100

103101
- name: nf-core modules install
104102
run: nf-core --log-file log.txt modules install fastqc --dir nf-core-testpipeline/ --force
105103

104+
- name: nf-core modules install gitlab
105+
run: nf-core --log-file log.txt modules --git-remote https://gitlab.com/nf-core/modules-test.git install fastqc --dir nf-core-testpipeline/
106+
107+
- name: nf-core modules list local
108+
run: nf-core --log-file log.txt modules list local --dir nf-core-testpipeline/
109+
110+
- name: nf-core modules list remote
111+
run: nf-core --log-file log.txt modules list remote
112+
113+
- name: nf-core modules list remote gitlab
114+
run: nf-core --log-file log.txt modules --git-remote https://gitlab.com/nf-core/modules-test.git list remote
115+
106116
- name: Upload log file artifact
107117
if: ${{ always() }}
108118
uses: actions/upload-artifact@v2

‎.github/workflows/create-test-wf.yml‎

Lines changed: 12 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,13 @@
11
name: Create a pipeline and test it
22
on: [push, pull_request]
33

4+
# Cancel if a newer run is started
5+
concurrency:
6+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
7+
cancel-in-progress: true
8+
49
env:
510
NXF_ANSI_LOG: false
6-
CAPSULE_LOG: none
711

812
jobs:
913
RunTestWorkflow:
@@ -12,14 +16,9 @@ jobs:
1216
NXF_ANSI_LOG: false
1317
strategy:
1418
matrix:
15-
# Nextflow versions
16-
include:
17-
# Test pipeline minimum Nextflow version
18-
- NXF_VER: "21.10.3"
19-
NXF_EDGE: ""
20-
# Test latest edge release of Nextflow
21-
- NXF_VER: ""
22-
NXF_EDGE: "1"
19+
NXF_VER:
20+
- "21.10.3"
21+
- "latest-everything"
2322
steps:
2423
- uses: actions/checkout@v2
2524
name: Check out source-code repository
@@ -35,18 +34,13 @@ jobs:
3534
pip install .
3635
3736
- name: Install Nextflow
38-
env:
39-
NXF_VER: ${{ matrix.NXF_VER }}
40-
# Uncomment only if the edge release is more recent than the latest stable release
41-
# See https://github.com/nextflow-io/nextflow/issues/2467
42-
# NXF_EDGE: ${{ matrix.NXF_EDGE }}
43-
run: |
44-
wget -qO- get.nextflow.io | bash
45-
sudo mv nextflow /usr/local/bin/
37+
uses: nf-core/setup-nextflow@v1
38+
with:
39+
version: ${{ matrix.NXF_VER }}
4640

4741
- name: Run nf-core/tools
4842
run: |
49-
nf-core --log-file log.txt create -n testpipeline -d "This pipeline is for testing" -a "Testing McTestface"
43+
nf-core --log-file log.txt create -n testpipeline -d "This pipeline is for testing" -a "Testing McTestface" --plain
5044
nextflow run nf-core-testpipeline -profile test,docker --outdir ./results
5145
5246
- name: Upload log file artifact

‎.github/workflows/deploy-pypi.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,11 @@ on:
33
release:
44
types: [published]
55

6+
# Cancel if a newer run is started
7+
concurrency:
8+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
9+
cancel-in-progress: true
10+
611
jobs:
712
build-n-publish:
813
runs-on: ubuntu-latest

‎.github/workflows/fix-linting.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,16 @@ jobs:
3838
# Override to remove the default --check flag so that we make changes
3939
options: "--color"
4040

41+
- name: Set up Python 3.8
42+
uses: actions/setup-python@v2
43+
with:
44+
python-version: 3.8
45+
- name: python-isort
46+
uses: isort/isort-action@v0.1.0
47+
with:
48+
isortVersion: "latest"
49+
requirementsFiles: "requirements.txt requirements-dev.txt"
50+
4151
- name: Commit & push changes
4252
run: |
4353
git config user.email "core@nf-co.re"

‎.github/workflows/lint-code.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,11 @@
11
name: Lint tools code formatting
22
on: [push, pull_request]
33

4+
# Cancel if a newer run is started
5+
concurrency:
6+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
7+
cancel-in-progress: true
8+
49
jobs:
510
EditorConfig:
611
runs-on: ubuntu-latest
@@ -58,3 +63,19 @@ jobs:
5863
Thanks again for your contribution!
5964
repo-token: ${{ secrets.GITHUB_TOKEN }}
6065
allow-repeats: false
66+
67+
isort:
68+
runs-on: ubuntu-latest
69+
steps:
70+
- name: Check out source-code repository
71+
uses: actions/checkout@v2
72+
73+
- name: Set up Python 3.8
74+
uses: actions/setup-python@v2
75+
with:
76+
python-version: 3.8
77+
- name: python-isort
78+
uses: isort/isort-action@v0.1.0
79+
with:
80+
isortVersion: "latest"
81+
requirementsFiles: "requirements.txt requirements-dev.txt"

‎.github/workflows/push_dockerhub_dev.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@ on:
55
push:
66
branches: [dev]
77

8+
# Cancel if a newer run is started
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
11+
cancel-in-progress: true
12+
813
jobs:
914
push_dockerhub:
1015
name: Push new Docker image to Docker Hub (dev)

‎.github/workflows/push_dockerhub_release.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@ on:
55
release:
66
types: [published]
77

8+
# Cancel if a newer run is started
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
11+
cancel-in-progress: true
12+
813
jobs:
914
push_dockerhub:
1015
name: Push new Docker image to Docker Hub (release)

‎.github/workflows/pytest.yml‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,17 @@ on:
55
push:
66
pull_request:
77

8-
# Uncomment if we need an edge release of Nextflow again
9-
# env: NXF_EDGE: 1
8+
# Cancel if a newer run is started
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
11+
cancel-in-progress: true
1012

1113
jobs:
1214
pytest:
1315
runs-on: ubuntu-latest
1416
strategy:
1517
matrix:
16-
python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]
18+
python-version: ["3.7", "3.8", "3.9", "3.10"]
1719

1820
steps:
1921
- uses: actions/checkout@v2
@@ -30,11 +32,9 @@ jobs:
3032
pip install -e .
3133
3234
- name: Install Nextflow
33-
env:
34-
CAPSULE_LOG: none
35-
run: |
36-
wget -qO- get.nextflow.io | bash
37-
sudo mv nextflow /usr/local/bin/
35+
uses: nf-core/setup-nextflow@v1
36+
with:
37+
version: "latest-everything"
3838

3939
- name: Test with pytest
4040
run: python3 -m pytest tests/ --color=yes --cov-report=xml --cov-config=.github/.coveragerc --cov=nf_core

0 commit comments

Comments
 (0)