-
Notifications
You must be signed in to change notification settings - Fork 342
Newcastle #961
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Newcastle #961
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -128,6 +128,7 @@ jobs: | |
| - "mssm" | ||
| - "munin" | ||
| - "nci_gadi" | ||
| - "newcastle" | ||
| - "nu_genomics" | ||
| - "nygc" | ||
| - "nyu_hpc" | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| params { | ||
| config_profile_description = 'Newcastle HPC cluster profile' | ||
| config_profile_contact = 'Chris Wyatt (@chriswyatt1), Fernando Duarte (@FernandoDuarteF)' | ||
| config_profile_url = '' | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Missing URL of cluster or instituteion |
||
| } | ||
|
|
||
| executor { | ||
| name = 'slurm' | ||
| } | ||
|
|
||
| process { | ||
| scratch = true | ||
| errorStrategy = { task.exitStatus in [137, 140] ? 'retry' : 'finish' } | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should already be in nf-core pipelines, but with an improved range: https://github.com/nf-core/tools/blob/75e643ad2c62cdc8ecd39aa56c0c314efd4b13e4/nf_core/pipeline-template/conf/base.config#L18 If you are adding this for non-official nf-core pipelines, I would recommend at least to match the nf-core one. |
||
| queue = 'defq' // maps to `#SBATCH -p defq` | ||
| clusterOptions = '--account=DESIM' // maps to `#SBATCH -A DESIM` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just to confirm, this works for every single user of the cluster? |
||
| } | ||
|
|
||
| singularity { | ||
| enabled = true | ||
| pullTimeout = '2 h' | ||
| autoMounts = true | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # nf-core/configs: Newcastle HPC Configuration | ||
|
|
||
| Configuration for the Newcastle high performance computing (HPC) cluster. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this University of Newcastle? |
||
|
|
||
| ## Setup | ||
|
|
||
| To install nextflow we first need to get the prequisite java from module avail: | ||
|
|
||
| `module load Java/17.0.6` | ||
|
|
||
| Then installing Nextflow using the standard install options: | ||
| https://www.nextflow.io/docs/latest/install.html | ||
|
|
||
| `curl -s https://get.nextflow.io | bash` | ||
|
|
||
| Make Nextflow executable: | ||
|
|
||
| `chmod +x nextflow` | ||
|
|
||
| Then move Nextflow into an executable path. For example: | ||
|
|
||
| `mkdir -p $HOME/bin/` | ||
| `mv nextflow $HOME/bin/` | ||
|
|
||
| To ensure this works every time you login to the cluster, put inside your `~/.bash_profile` the java install: | ||
|
|
||
| `nano ~/.bash_profile` | ||
|
|
||
| Then paste in the line for the java install: | ||
|
|
||
| ``` | ||
| #Java module add v17: | ||
| module load Java/17.0.6 | ||
| ``` | ||
|
|
||
| Exit nano with, control-X, then y to save, then enter to confirm | ||
|
|
||
| ## Run | ||
|
|
||
| To use this profile, simply pass Nextflow the -profile newcastle flag eg. | ||
|
|
||
| ```bash | ||
| nextflow run nf-core/_pipeline_ -profile newcastle --outdir results | ||
| ``` | ||
|
|
||
| In case of issues, please feel free to reach out to one of the maintainers (Chris Wyatt or Fernando Duarte) at ecoflow.ucl@gmail.com . We are based at UCL, but helped set up this config, and are happy to help. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You have no resource limits specified, are there no limits?