Skip to content

Commit 06f545b

Browse files
jonathangoekejonathangoeke
authored
Merge pull request #11 from samuell/add-scipipe
Implement SciPipe example workflow
2 parents e08e2e0 + f995d6f commit 06f545b

File tree

7 files changed

+170
-0
lines changed

7 files changed

+170
-0
lines changed

‎.github/workflows/scipipe.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: Test-SciPipe
2+
3+
on:
4+
push:
5+
branches: [ master, add-scipipe ]
6+
pull_request:
7+
branches: [ master ]
8+
9+
jobs:
10+
Testing:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- name: Check out the code
14+
uses: actions/checkout@v2
15+
16+
- name: Setup Go
17+
uses: actions/setup-go@v2
18+
with:
19+
go-version: '1.16.6'
20+
21+
- name: Install bioinformatics tools
22+
run: sudo apt-get install salmon fastqc -y
23+
24+
- name: Tests
25+
run: |
26+
cd scipipe && go run example.go \
27+
-left $PWD/../test_data/reads_1.fq.gz \
28+
-right $PWD/../test_data/reads_2.fq.gz \
29+
-ref $PWD/../test_data/transcriptome.fa \
30+
-outdir $PWD/results

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ Each workflow manager folder in this repository has a README detailing how to ru
2424
- [Galaxy](galaxy)
2525
- [Nextflow](nextflow)
2626
- [Snakemake](snakemake)
27+
- [SciPipe](scipipe)
2728
- [WDL](wdl)
2829

2930
## Online Documentation for Workflow managers
@@ -51,3 +52,4 @@ We would like to thank the following people for their contribution to this repos
5152
- [Rob Patro](https://github.com/rob-p) for creating the [RNA-Seq test data set](test_data)
5253
- [Paolo Di Tommaso](https://github.com/pditommaso) for the [Nextflow workflow](nextflow)
5354
- [Johannes Köster](https://github.com/johanneskoester) for the [Snakemake workflow](snakemake)
55+
- [Samuel Lampa](https://github.com/samuell) for the [SciPipe workflow](scipipe)

‎scipipe/README.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
## About SciPipe
2+
3+
SciPipe is a workflow library implemented as a programming library in Go.
4+
See [scipipe.org](https://scipipe/org) for more information!
5+
6+
## Training material and documentation
7+
8+
- [Link to tutorial](https://scipipe.org/writing_workflows/)
9+
- [Link to online documentation](https://scipipe.org)
10+
- [A basic tutorial as a screencast](https://www.youtube.com/watch?v=hi0Uqwddrtg)
11+
- [Case study workflows for the SciPipe paper](https://github.com/pharmbio/scipipe-demo)
12+
13+
## Community-developed workflows in SciPipe
14+
15+
- "Reproducible Probabilistic Target Profiles"
16+
- Repository: https://github.com/pharmbio/ptp-project
17+
- Description: A workflow developed by the workflow tool authors, for building
18+
target binding profiles, using QSAR and SVM (see the [paper](https://doi.org/10.3389/fphar.2018.01256) for more details).
19+
- Kleuren SciPipe workflow
20+
- Repository: https://github.com/Colelyman/kleuren-scipipe-workflow
21+
- Description: A scipipe workflow for building the necessary data structures
22+
for running kleuren.
23+
24+
## Running the proof of concept SciPipe pipeline
25+
26+
1. Install the Go toolchain by following [this page](https://go.dev/learn/) for
27+
the specific instructions for your operating system.
28+
- Note that on Windows, you need to have a POSIX-compliant bash environment
29+
to run Scipipe, such as Windows Subsystem for Linux (WSL). Find more
30+
information about [installing WSL here](https://docs.microsoft.com/en-us/windows/wsl/install-win10)
31+
2. Go into the scipipe directory:
32+
```bash
33+
cd scipipe
34+
```
35+
3. Run the workflow using the `run.sh` bash script:
36+
```bash
37+
./run.sh
38+
```
39+
Alternatively you can run the full command directly:
40+
```bash
41+
go run example.go -ref ../test_data/transcriptome.fa -left ../test_data/reads_1.fq.gz -right ../test_data/reads_2.fq.gz -outdir results
42+
```
43+
44+
Done!
45+
46+
## Notes and Contribution
47+
48+
This pipeline is a minimal example of using X. We welcome contributions to the
49+
documentation and workflow, please create an issue or submit a pull request!
50+
51+
## How to cite SciPipe
52+
53+
Please cite SciPipe as:
54+
55+
> Lampa, S., Dahlö, M., Alvarsson, J., & Spjuth, O. (2019). SciPipe: A workflow
56+
> library for agile development of complex and dynamic bioinformatics pipelines.
57+
> GigaScience, 8(5), giz044. DOI: [10.1093/gigascience/giz044](https://doi.org/10.1093/gigascience/giz044)

‎scipipe/example.go‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
package main
2+
3+
import (
4+
"flag"
5+
"os"
6+
7+
sp "github.com/scipipe/scipipe"
8+
spc "github.com/scipipe/scipipe/components"
9+
)
10+
11+
func main() {
12+
// ----------------------------------------------------------------
13+
// Define input parameters to the workflow as command line flags
14+
// ----------------------------------------------------------------
15+
refPath := flag.String("ref", "", "Reference file")
16+
leftPath := flag.String("left", "", "The first reads file")
17+
rightPath := flag.String("right", "", "The second reads file")
18+
outDirPath := flag.String("outdir", "", "Output directory")
19+
flag.Parse()
20+
21+
// ----------------------------------------------------------------
22+
// Ensure flags are set
23+
// ----------------------------------------------------------------
24+
if *refPath == "" || *leftPath == "" || *rightPath == "" || *outDirPath == "" {
25+
flag.PrintDefaults()
26+
os.Exit(1)
27+
}
28+
29+
// ----------------------------------------------------------------
30+
// Initialize workflow with 4 workers
31+
// ----------------------------------------------------------------
32+
wf := sp.NewWorkflow("example-workflow", 4)
33+
34+
// We need to create workflow processes to feed the files to downstream
35+
// processes as IPs (information packets)
36+
refSrc := spc.NewFileSource(wf, "ref-file", *refPath)
37+
leftSrc := spc.NewFileSource(wf, "left", *leftPath)
38+
rightSrc := spc.NewFileSource(wf, "right", *rightPath)
39+
40+
// ----------------------------------------------------------------
41+
// Salmon indexing process
42+
// ----------------------------------------------------------------
43+
salmonIndex := wf.NewProc("salmon-index", "salmon index -t {i:ref} -i {o:index}")
44+
salmonIndex.In("ref").From(refSrc.Out())
45+
salmonIndex.SetOut("index", "data/salmon-index")
46+
47+
// ----------------------------------------------------------------
48+
// Transcriptome alignment and quantification using Salmon
49+
// ----------------------------------------------------------------
50+
salmonAlignQuant := wf.NewProc("salmon-align-quant", "salmon quant -i {i:index} -l A -1 '{i:left}' -2 '{i:right}' --validateMappings -o {o:quant}")
51+
salmonAlignQuant.In("index").From(salmonIndex.Out("index"))
52+
salmonAlignQuant.In("left").From(leftSrc.Out())
53+
salmonAlignQuant.In("right").From(rightSrc.Out())
54+
salmonAlignQuant.SetOut("quant", *outDirPath)
55+
56+
// ----------------------------------------------------------------
57+
// Quality control using FastQC
58+
// ----------------------------------------------------------------
59+
fastQC := wf.NewProc("fastqc", "mkdir -p {o:qcdir} && fastqc --quiet '{i:left}' '{i:right}' --outdir {o:qcdir}")
60+
fastQC.In("left").From(leftSrc.Out())
61+
fastQC.In("right").From(rightSrc.Out())
62+
fastQC.SetOut("qcdir", "data/qc")
63+
64+
// ----------------------------------------------------------------
65+
// Run workflow
66+
// ----------------------------------------------------------------
67+
wf.Run()
68+
}

‎scipipe/go.mod‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
module example
2+
3+
go 1.16
4+
5+
require github.com/scipipe/scipipe v0.10.2

‎scipipe/go.sum‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
github.com/scipipe/scipipe v0.10.2 h1:crXD1gGh/LuBfWfT4CdXcRFtPjem5weyXN03BDfVOuU=
2+
github.com/scipipe/scipipe v0.10.2/go.mod h1:Nwof+Uimtam7GTpkU6cAf/EOnqvxcOVFytjnYU5I3vY=

‎scipipe/run.sh‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
#!/bin/bash
2+
go run example.go \
3+
-ref ../test_data/transcriptome.fa \
4+
-left ../test_data/reads_1.fq.gz \
5+
-right ../test_data/reads_2.fq.gz \
6+
-outdir results

0 commit comments

Comments
 (0)