Merge pull request #7 from tobyhodges/gh-pages

Transferred existing user guide material

Author: mr-c

_episodes/01-introduction.md

@@ -28,5 +28,3 @@ technologies such as Docker, be used with CWL implementations from different
 vendors, and is well suited for describing large-scale workflows in cluster,
 cloud and high performance computing environments where tasks are scheduled in
 parallel across many nodes.
-
-{% include links.md %}

_episodes/02-1st-example.md

@@ -0,0 +1,76 @@
+---
+title: "First Example"
+teaching: 5
+exercises: 0
+questions:
+- "How do I wrap a simple command line tool?"
+objectives:
+- "Learn the basic structure of a CWL description."
+keypoints:
+- "CWL documents are written in YAML and/or JSON."
+- "The command called is specified with `baseCommand`."
+- "Each expected input is described in the `inputs` section."
+- "Input values are specified in a separate YAML file."
+- "The tool description and input files are provided as arguments to a CWL runner."
+---
+The simplest "hello world" program.  This accepts one input parameter, writes a message to the terminal or job log, and produces no permanent output. CWL documents are written in [JSON][json] or [YAML][yaml], or a mix of the two.
+
+
+*1st-tool.cwl*
+```
+{% include cwl/1st-tool.cwl %}
+```
+
+Use a YAML object in a separate file to describe the input of a run:
+
+*echo-job.yml*
+```
+{% include cwl/echo-job.yml %}
+```
+
+Now invoke `cwl-runner` with the tool wrapper and the input object on the command line:
+
+```
+$ cwl-runner 1st-tool.cwl echo-job.yml
+[job 1st-tool.cwl] /tmp/tmpmM5S_1$ echo \
+    'Hello world!'
+Hello world!
+[job 1st-tool.cwl] completed success
+{}
+Final process status is success
+
+```
+
+What's going on here?  Let's break it down:
+
+```
+cwlVersion: v1.0
+class: CommandLineTool
+```
+
+The `cwlVersion` field indicates the version of the CWL spec used by the document.  The `class` field indicates this document describes a command line tool.
+
+```
+baseCommand: echo
+```
+
+The `baseCommand` provides the name of program that will actually run (`echo`)
+
+```
+inputs:
+  message:
+    type: string
+      inputBinding:
+        position: 1
+```
+
+The `inputs` section describes the inputs of the tool.  This is a list of input parameters and each parameter includes an identifier, a data type, and optionally an `inputBinding` which describes how this input parameter should appear on the command line.  In this example, the `position` field indicates where it should appear on the command line.
+
+```
+outputs: []
+```
+
+This tool has no formal output, so the `outputs` section is an empty list.
+
+[json]: http://json.org
+[yaml]: http://yaml.org

_episodes/03-input.md

@@ -0,0 +1,141 @@
+---
+title: "Essential Input Parameters"
+teaching: 10
+exercises: 0
+questions:
+- "How do I describe inputs to a command?"
+- "How do I specify the order in which inputs appear in a command?"
+objectives:
+- "Learn how to describe and handle input parameters and files to a tool."
+keypoints:
+- "Inputs are described in the `inputs` section of a CWL description."
+- "Files should be described with `class: File`."
+- "You can use the `inputBinding` section to describe where and how an input
+appears in the command."
+---
+
+The `inputs` of a tool is a list of input parameters that control how to
+run the tool.  Each parameter has an `id` for the name of parameter, and
+`type` describing what types of values are valid for that parameter.
+
+Available primitive types are *string*, *int*, *long*, *float*, *double*,
+and *null*; complex types are *array* and *record*; in addition there are
+special types *File*, *Directory* and *Any*.
+
+The following example demonstrates some input parameters with different
+types and appearing on the command line in different ways:
+
+
+*inp.cwl*
+
+~~~
+{% include cwl/inp.cwl %}
+~~~
+
+*inp-job.yml*
+
+```
+{% include cwl/inp-job.yml %}
+```
+
+Notice that "example_file", as a `File` type, must be provided as an
+object with the fields `class: File` and `path`.
+
+Next, create a whale.txt and invoke `cwl-runner` with the tool wrapper and the
+input object on the command line:
+
+```
+$ touch whale.txt
+$ cwl-runner inp.cwl inp-job.yml
+[job inp.cwl] /tmp/tmpzrSnfX$ echo \
+    -f \
+    -i42 \
+    --example-string \
+    hello \
+    --file=/tmp/tmpRBSHIG/stg979b6d24-d50a-47e3-9e9e-90097eed2cbc/whale.txt
+-f -i42 --example-string hello --file=/tmp/tmpRBSHIG/stg979b6d24-d50a-47e3-9e9e-90097eed2cbc/whale.txt
+[job inp.cwl] completed success
+{}
+Final process status is success
+```
+> ## Where did those `/tmp` paths come from?
+>
+> The CWL reference runner (cwltool) and other runners create temporary
+> directories with symbolic ("soft") links to your input files to ensure that
+> the tools aren't accidently accessing files that were not explicitly
+> specified
+{: .callout}
+
+The field `inputBinding` is optional and indicates whether and how the
+input parameter should be appear on the tool's command line.  If
+`inputBinding` is missing, the parameter does not appear on the command
+line.  Let's look at each example in detail.
+
+```
+example_flag:
+  type: boolean
+  inputBinding:
+    position: 1
+    prefix: -f
+```
+
+Boolean types are treated as a flag.  If the input parameter
+"example_flag" is "true", then `prefix` will be added to the
+command line.  If false, no flag is added.
+
+```
+example_string:
+  type: string
+  inputBinding:
+    position: 3
+    prefix: --example-string
+```
+
+String types appear on the command line as literal values.  The `prefix`
+is optional, if provided, it appears as a separate argument on the
+command line before the parameter .  In the example above, this is
+rendered as `--example-string hello`.
+
+```
+example_int:
+  type: int
+  inputBinding:
+    position: 2
+    prefix: -i
+    separate: false
+```
+
+Integer (and floating point) types appear on the command line with
+decimal text representation.  When the option `separate` is false (the
+default value is true), the prefix and value are combined into a single
+argument.  In the example above, this is rendered as `-i42`.
+
+
+```
+example_file:
+  type: File?
+  inputBinding:
+    prefix: --file=
+    separate: false
+    position: 4
+```
+
+File types appear on the command line as the path to the file.  When the
+parameter type ends with a question mark `?` it indicates that the
+parameter is optional.  In the example above, this is rendered as
+`--file=/tmp/random/path/whale.txt`.  However, if the "example_file"
+parameter were not provided in the input, nothing would appear on the
+command line.
+
+Input files are read-only.  If you wish to update an input file, you must
+[first copy it to the output directory]({{ page.root }}/15-staging/).
+
+The value of `position` is used to determine where parameter should
+appear on the command line.  Positions are relative to one another, not
+absolute.  As a result, positions do not have to be sequential, three
+parameters with positions 1, 3, 5 will result in the same command
+line as 1, 2, 3.  More than one parameter can have the same position
+(ties are broken using the parameter name), and the position field itself
+is optional.  The default position is 0.
+
+The `baseCommand` field will always appear in the final command line before the parameters.

_episodes/04-output.md

@@ -0,0 +1,75 @@
+---
+title: "Returning Output Files"
+teaching: 10
+exercises: 0
+questions:
+- "How do I describe outputs from a command?"
+objectives:
+- "Learn how to describe and handle outputs from a tool."
+keypoints:
+- "Outputs are described in the `outputs` section of a CWL description."
+- "The field `outputBinding` describes how to to set the value of each
+output parameter."
+- "Wildcards are allowed in the `glob` field."
+---
+The `outputs` of a tool is a list of output parameters that should be
+returned after running the tool.  Each parameter has an `id` for the name
+of parameter, and `type` describing what types of values are valid for
+that parameter.
+
+When a tool runs under CWL, the starting working directory is the
+designated output directory.  The underlying tool or script must record
+its results in the form of files created in the output directory.  The
+output parameters returned by the CWL tool are either the output files
+themselves, or come from examining the content of those files.
+
+*tar.cwl*
+
+```
+{% include cwl/tar.cwl %}
+```
+
+*tar-job.yml*
+
+```
+{% include cwl/tar-job.yml %}
+```
+
+Next, create a tar file for the example and invoke `cwl-runner` with the tool
+wrapper and the input object on the command line:
+
+```
+$ touch hello.txt && tar -cvf hello.tar hello.txt
+$ cwl-runner tar.cwl tar-job.yml
+[job tar.cwl] /tmp/tmpqOeawQ$ tar \
+    xf \
+    /tmp/tmpGDk8Y1/stg80bbad20-494d-47af-8075-dffc32df03a3/hello.tar
+[job tar.cwl] completed success
+{
+    "example_out": {
+        "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709", 
+        "basename": "hello.txt", 
+        "nameroot": "hello", 
+        "nameext": ".txt", 
+        "location": "file:///home/me/cwl/user_guide/hello.txt", 
+        "path": "/home/me/cwl/user_guide/hello.txt", 
+        "class": "File", 
+        "size": 0
+    }
+}
+Final process status is success
+```
+
+The field `outputBinding` describes how to to set the value of each
+output parameter.
+
+```
+outputs:
+  example_out:
+    type: File
+    outputBinding:
+      glob: hello.txt
+```
+
+The `glob` field consists of the name of a file in the output directory.
+If you don't know name of the file in advance, you can use a wildcard pattern like `*.txt`.

_episodes/05-stdout.md

@@ -0,0 +1,51 @@
+---
+title: "Capturing Standard Output"
+teaching: 10
+exercises: 0
+questions:
+- "How do I capture a tool's standard output stream?"
+objectives:
+- "Learn how to capture streamed output from a tool."
+keypoints:
+- "Use the `stdout` field to specify a filename to capture streamed output."
+- "The corresponding output parameter must have `type: stdout`."
+---
+To capture a tool's standard output stream, add the `stdout` field with
+the name of the file where the output stream should go.  Then add `type:
+stdout` on the corresponding output parameter.
+
+*stdout.cwl*
+
+```
+{% include cwl/stdout.cwl %}
+```
+
+*echo-job.yml*
+
+```
+{% include cwl/echo-job.yml %}
+```
+
+Now invoke `cwl-runner` providing the tool wrapper and the input object
+on the command line:
+
+```
+$ cwl-runner stdout.cwl echo-job.yml
+[job stdout.cwl] /tmp/tmpE0gTz7$ echo \
+    'Hello world!' > /tmp/tmpE0gTz7/output.txt
+[job stdout.cwl] completed success
+{
+    "output": {
+        "checksum": "sha1$47a013e660d408619d894b20806b1d5086aab03b",
+        "basename": "output.txt",
+        "nameroot": "output",
+        "nameext": ".txt",
+        "location": "file:///home/me/cwl/user_guide/output.txt",
+        "path": "/home/me/cwl/user_guide/output.txt",
+        "class": "File",
+        "size": 13
+    }
+}
+Final process status is success
+
+```