Skip to content

Commit 797fa95

Browse files
author
Peter Amstutz
committed
Squashed 'cwltool/schemas/' changes from 4e24484..d674994
d674994 Merge branch 'master' into draft-4 bef607e Fix draft-3 conformance tests missing 'cwlVersion'. The 'cwlVersion' field is now a @vocab instead of an @id. be35388 Merge branch 'master' of github.com:common-workflow-language/common-workflow-language 6cf9db8 Add workflow example to user guide. Add missing files. git-subtree-dir: cwltool/schemas git-subtree-split: d674994
1 parent 8665ea5 commit 797fa95

15 files changed

+262
-14
lines changed

‎draft-3/Process.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -510,7 +510,7 @@ $graph:
510510
doc: "CWL document version"
511511
jsonldPredicate:
512512
"_id": "cwl:cwlVersion"
513-
"_type": "@id"
513+
"_type": "@vocab"
514514

515515
- name: InlineJavascriptRequirement
516516
type: record

‎draft-3/UserGuide.yml

Lines changed: 181 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@
33
doc:
44
- $include: userguide-intro.md
55

6+
- |
7+
# Wrapping Command Line Tools
8+
69
- |
710
## First example
811
@@ -11,9 +14,9 @@
1114
output. CWL documents are written in [JSON](http://json.org) or
1215
[YAML](http://yaml.org), or a mix of the two.
1316
14-
*1st.cwl*
17+
*1st-tool.cwl*
1518
```
16-
- $include: examples/1st.cwl
19+
- $include: examples/1st-tool.cwl
1720
- |
1821
```
1922
@@ -29,7 +32,7 @@
2932
command line:
3033
3134
```
32-
$ cwl-runner 1st.cwl echo-job.yml
35+
$ cwl-runner 1st-tool.cwl echo-job.yml
3336
[job 140199012414352] $ echo 'Hello world!'
3437
Hello world!
3538
Final process status is success
@@ -308,7 +311,7 @@
308311
Invoke `cwl-runner` with the tool wrapper and the input object on the
309312
command line:
310313
```
311-
$ cwl-runner example-tar-param.cwl example-tar-param-job.yml
314+
$ cwl-runner tar-param.cwl tar-param-job.yml
312315
[job 139868145165200] $ tar xf /home/example/hello.tar goodbye.txt
313316
Final process status is success
314317
{
@@ -475,8 +478,7 @@
475478
476479
```
477480
$ cwl-runner array-inputs.cwl array-inputs-job.yml
478-
/home/peter/work/cwl-venv/bin/cwltool 1.0.20160325210917
479-
[job 140334923640912] /home/peter/work/common-workflow-language/draft-3/examples$ echo -A one two three -B=four -B=five -B=six -C=seven,eight,nine
481+
[job 140334923640912] /home/example$ echo -A one two three -B=four -B=five -B=six -C=seven,eight,nine
480482
-A one two three -B=four -B=five -B=six -C=seven,eight,nine
481483
Final process status is success
482484
{}
@@ -492,6 +494,48 @@
492494
You can specify arrays of arrays, arrays of records, and other complex
493495
types.
494496
497+
- |
498+
## Array outputs
499+
500+
You can also capture multiple output files into an array of files using `glob`.
501+
502+
*array-outputs.cwl*
503+
```
504+
- $include: examples/array-outputs.cwl
505+
- |
506+
```
507+
508+
*array-outpust-job.yml*
509+
```
510+
- $include: examples/array-outputs-job.yml
511+
- |
512+
```
513+
514+
Now invoke `cwl-runner` providing the tool wrapper and the input object
515+
on the command line:
516+
517+
```
518+
$ cwl-runner array-outputs.cwl array-outputs-job.yml
519+
[job 140190876078160] /home/example$ touch foo.txt bar.dat baz.txt
520+
Final process status is success
521+
{
522+
"output": [
523+
{
524+
"size": 0,
525+
"path": "/home/peter/work/common-workflow-language/draft-3/examples/foo.txt",
526+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
527+
"class": "File"
528+
},
529+
{
530+
"size": 0,
531+
"path": "/home/peter/work/common-workflow-language/draft-3/examples/baz.txt",
532+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
533+
"class": "File"
534+
}
535+
]
536+
}
537+
```
538+
495539
- |
496540
## Record inputs, dependent and mutually exclusive parameters
497541
@@ -672,14 +716,143 @@
672716
673717
```
674718
$ cwl-runner linkfile.cwl arguments-job.yml
675-
[job 139928309171664] /home/example$ docker run -i --volume=/home/peter/work/common-workflow-language/draft-3/examples/Hello.java:/var/lib/cwl/job557617295_examples/Hello.java:ro --volume=/home/peter/work/common-workflow-language/draft-3/examples:/var/spool/cwl:rw --volume=/tmp/tmpmNbApw:/tmp:rw --workdir=/var/spool/cwl --read-only=true --net=none --user=1001 --rm --env=TMPDIR=/tmp java:7 javac Hello.java
719+
[job 139928309171664] /home/example$ docker run -i --volume=/home/example/Hello.java:/var/lib/cwl/job557617295_examples/Hello.java:ro --volume=/home/example:/var/spool/cwl:rw --volume=/tmp/tmpmNbApw:/tmp:rw --workdir=/var/spool/cwl --read-only=true --net=none --user=1001 --rm --env=TMPDIR=/tmp java:7 javac Hello.java
676720
Final process status is success
677721
{
678722
"classfile": {
679723
"size": 416,
680-
"path": "/home/peter/work/common-workflow-language/draft-3/examples/Hello.class",
724+
"path": "/home/example/Hello.class",
681725
"checksum": "sha1$2f7ac33c1f3aac3f1fec7b936b6562422c85b38a",
682726
"class": "File"
683727
}
684728
}
685729
```
730+
731+
- |
732+
# Writing Workflows
733+
734+
## First workflow
735+
736+
This workflow extracts a java source file from a tar file and then
737+
compiles it.
738+
739+
*1st-workflow.cwl*
740+
```
741+
- $include: examples/1st-workflow.cwl
742+
- |
743+
```
744+
745+
Use a JSON object in a separate file to describe the input of a run:
746+
747+
*1st-workflow-job.yml*
748+
```
749+
- $include: examples/1st-workflow-job.yml
750+
- |
751+
```
752+
753+
Now invoke `cwl-runner` with the tool wrapper and the input object on the
754+
command line:
755+
756+
```
757+
$ cwl-runner 1st-workflow.cwl 1st-workflow-job.yml
758+
[job untar] /tmp/tmp94qFiM$ tar xf /home/example/hello.tar Hello.java
759+
[step untar] completion status is success
760+
[job compile] /tmp/tmpu1iaKL$ docker run -i --volume=/tmp/tmp94qFiM/Hello.java:/var/lib/cwl/job301600808_tmp94qFiM/Hello.java:ro --volume=/tmp/tmpu1iaKL:/var/spool/cwl:rw --volume=/tmp/tmpfZnNdR:/tmp:rw --workdir=/var/spool/cwl --read-only=true --net=none --user=1001 --rm --env=TMPDIR=/tmp java:7 javac -d /var/spool/cwl /var/lib/cwl/job301600808_tmp94qFiM/Hello.java
761+
[step compile] completion status is success
762+
[workflow 1st-workflow.cwl] outdir is /home/example
763+
Final process status is success
764+
{
765+
"classout": {
766+
"path": "/home/example/Hello.class",
767+
"checksum": "sha1$2f7ac33c1f3aac3f1fec7b936b6562422c85b38a",
768+
"class": "File",
769+
"size": 416
770+
}
771+
}
772+
```
773+
774+
What's going on here? Let's break it down:
775+
776+
```
777+
cwlVersion: cwl:draft-3
778+
class: Workflow
779+
```
780+
781+
The 'cwlVersion` field indicates the version of the CWL spec used by the
782+
document. The `class` field indicates this document describes a workflow.
783+
784+
785+
```
786+
inputs:
787+
- id: inp
788+
type: File
789+
- id: ex
790+
type: string
791+
```
792+
793+
The `inputs` section describes the inputs of the workflow. This is a
794+
list of input parameters where each parameter consists of an identifier
795+
and a data type. These parameters can be used as sources for input to
796+
specific workflows steps.
797+
798+
```
799+
outputs:
800+
- id: classout
801+
type: File
802+
source: "#compile/classfile"
803+
```
804+
805+
The `outputs` section describes the outputs of the workflow. This is a
806+
list of output parameters where each parameter consists of an identifier
807+
and a data type. The `source` connects the output parameter `classfile`
808+
of the `compile` step to the workflow output parameter `classout`.
809+
810+
```
811+
steps:
812+
- id: untar
813+
run: tar-param.cwl
814+
inputs:
815+
- id: tarfile
816+
source: "#inp"
817+
- id: extractfile
818+
source: "#ex"
819+
outputs:
820+
- id: example_out
821+
```
822+
823+
The `steps` section describes the actual steps of the workflow. In this
824+
example, the first step extracts a file from a tar file, and the second
825+
step compiles the file from the first step using the java compiler.
826+
Workflow steps are not necessarily run in the order they are listed,
827+
instead the order is determined by the dependencies between steps (using
828+
`source`). In addition, workflow steps which do not depend on one
829+
another may run in parallel.
830+
831+
The first step, `untar` runs `tar-param.cwl` (described previously in
832+
[Parameter references](#Parameter_references)). This tool has two input
833+
parameters, `tarfile` and `extractfile` and one output parameter
834+
`example_out`.
835+
836+
The `inputs` section of the workflow step connects these two input
837+
parameters to the inputs of the workflow, `#inp` and `#ex` using
838+
`source`. This means that when the workflow step is executed, the values
839+
assigned to `#inp` and `#ex` will be used for the parameters `tarfile`
840+
and `extractfile` in order to run the tool.
841+
842+
The `outputs` section of the workflow step lists the output parameters
843+
that are expected from the tool.
844+
845+
```
846+
- id: compile
847+
run: arguments.cwl
848+
inputs:
849+
- id: src
850+
source: "#untar/example_out"
851+
outputs:
852+
- id: classfile
853+
```
854+
855+
The second step `compile` depends on the results from the first step by
856+
connecting the input parameter `src` to the output parameter of `untar`
857+
using `#untar/example_out`. The output of this step `classfile` is
858+
connected to the `outputs` section for the Workflow, described above.

‎draft-3/draft-3/record-output.cwl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
class: CommandLineTool
2+
cwlVersion: draft-3
23
requirements:
34
- class: ShellCommandRequirement
45
inputs:

‎draft-3/draft-3/rename.cwl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
class: CommandLineTool
2+
cwlVersion: draft-3
23
baseCommand: "true"
34
requirements:
45
- class: CreateFileRequirement

‎draft-3/draft-3/schemadef-tool.cwl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/usr/bin/env cwl-runner
22
class: CommandLineTool
3-
cwlVersion: cwl:draft-3
3+
cwlVersion: draft-3
44
requirements:
55
- $import: schemadef-type.yml
66
- class: InlineJavascriptRequirement

‎draft-3/draft-3/wc3-tool.cwl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
#!/usr/bin/env cwl-runner
22
class: CommandLineTool
3+
cwlVersion: draft-3
34

45
requirements:
56
- class: InlineJavascriptRequirement

‎draft-3/examples/1st.cwl renamed to ‎draft-3/examples/1st-tool.cwl

File renamed without changes.

‎draft-3/examples/1st-workflow-job.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
inp:
2+
class: File
3+
path: hello.tar
4+
ex: Hello.java

‎draft-3/examples/1st-workflow.cwl

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
cwlVersion: cwl:draft-3
2+
class: Workflow
3+
inputs:
4+
- id: inp
5+
type: File
6+
- id: ex
7+
type: string
8+
9+
outputs:
10+
- id: classout
11+
type: File
12+
source: "#compile/classfile"
13+
14+
steps:
15+
- id: untar
16+
run: tar-param.cwl
17+
inputs:
18+
- id: tarfile
19+
source: "#inp"
20+
- id: extractfile
21+
source: "#ex"
22+
outputs:
23+
- id: example_out
24+
25+
- id: compile
26+
run: arguments.cwl
27+
inputs:
28+
- id: src
29+
source: "#untar/example_out"
30+
outputs:
31+
- id: classfile

‎draft-3/examples/array-outputs-job.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
touchfiles:
2+
- foo.txt
3+
- bar.dat
4+
- baz.txt

0 commit comments

Comments
 (0)