Merge pull request #47 from common-workflow-language/map_syntax

first draft of a map<> explainer

Author: mr-c

CommandLineTool.yml

@@ -105,6 +105,9 @@ $graph:
           now be calculated from a CWL Expression.
         * The exit code of a CommandLineTool invocation is now
           available to expressions in `outputEval` as `runtime.exitCode`
+        * [Better explain](#map) the `map<…>` notation that has existed since v1.0.
+        * Fixed schema error where the `type` field inside the `inputs` and
+          `outputs` field was incorrectly listed as optional.
 
       See also the [CWL Workflow Description, v1.1.0-dev1 changelog](Workflow.html#Changelog).
 
@@ -449,7 +452,6 @@ $graph:
   fields:
     - name: type
       type:
-        - "null"
         - CWLType
         - stdin
         - CommandInputRecordSchema
@@ -484,7 +486,6 @@ $graph:
   fields:
     - name: type
       type:
-        - "null"
         - CWLType
         - stdout
         - stderr

Workflow.yml

@@ -88,6 +88,10 @@ $graph:
         * [Added](#LoadListingRequirement) `LoadListingRequirement`
           and [loadListing](#LoadContents) to control whether and how
           `Directory` listings should be loaded for use in expressions.
+        * [Better explain](#map) the `map<…>` notation that has existed since v1.0.
+        * Fixed schema error where the `type` field inside the `inputs` and
+          `outputs` fields for both `Workflow` and `ExpressionTool` was
+           incorrectly listed as optional.
 
       See also the [CWL Command Line Tool Description, v1.1.0-dev1 changelog](CommandLineTool.html#Changelog).
 
@@ -110,7 +114,6 @@ $graph:
   fields:
     - name: type
       type:
-        - "null"
         - CWLType
         - OutputRecordSchema
         - OutputEnumSchema
@@ -139,7 +142,6 @@ $graph:
   fields:
     - name: type
       type:
-        - "null"
         - CWLType
         - InputRecordSchema
         - InputEnumSchema
@@ -235,7 +237,6 @@ $graph:
         If not specified, the default method is "merge_nested".
     - name: type
       type:
-        - "null"
         - CWLType
         - OutputRecordSchema
         - OutputEnumSchema

concepts.md

@@ -91,6 +91,185 @@ An implementation may formally validate the structure of a CWL document using
 SALAD schemas located at
 https://github.com/common-workflow-language/common-workflow-language/tree/master/v1.1.0-dev1
 
+### map
+
+Note: This section is non-normative.
+> type: array<ComplexType> |  
+> map<`key_field`, ComplexType>
+
+The above syntax in the CWL specifications means there are two or more ways to write the given value.
+
+Option one is a array and is the most verbose option.
+
+Option one generic example:
+```
+some_cwl_field:
+  - key_field: a_complex_type1
+    field2: foo
+    field3: bar
+  - key_field: a_complex_type2
+    field2: foo2
+    field3: bar2
+  - key_field: a_complex_type3
+```
+
+Option one specific example using [Workflow](Workflow.html#Workflow).[inputs](Workflow.html#WorkflowInputParameter):
+> array<InputParameter> |  
+> map<`id`, `type` | InputParameter>
+
+
+```
+inputs:
+  - id: workflow_input01
+    type: string
+  - id: workflow_input02
+    type: File
+    format: http://edamontology.org/format_2572
+```
+
+Option two is enabled by the `map<…>` syntax. Instead of an array of entries we
+use a mapping, where one field of the `ComplexType` (here named `key_field`)
+becomes the key in the map, and its value is the rest of the `ComplexType`
+without the key field. If all of the other fields of the `ComplexType` are
+optional and unneeded, then we can indicate this with an empty mapping as the
+value: `a_complex_type3: {}`
+
+Option two generic example:
+```
+some_cwl_field:
+  a_complex_type1:  # this was the "key_field" from above
+    field2: foo
+    field3: bar
+  a_complex_type2:
+    field2: foo2
+    field3: bar2
+  a_complex_type3: {}  # we accept the defualt values for "field2" and "field3"
+```
+
+Option two specific example using [Workflow](Workflow.html#Workflow).[inputs](Workflow.html#WorkflowInputParameter):
+> array&lt;InputParameter&gt; |  
+> map&lt;`id`, `type` | InputParameter&gt;
+
+
+```
+inputs:
+  workflow_input01:
+    type: string
+  workflow_input02:
+    type: File
+    format: http://edamontology.org/format_2572
+```
+
+Option two specific example using [SoftwareRequirement](#SoftwareRequirement).[packages](#SoftwarePackage):
+> array&lt;SoftwarePackage&gt; |  
+> map&lt;`package`, `specs` | SoftwarePackage&gt;
+
+
+```
+hints:
+  SoftwareRequirement:
+    packages:
+      sourmash:
+        specs: [ https://doi.org/10.21105/joss.00027 ]
+      screed:
+        version: [ "1.0" ]
+      python: {}
+```
+`
+Sometimes we have a third and even more compact option denoted like this:
+> type: array&lt;ComplexType&gt; |  
+> map&lt;`key_field`, `field2` | ComplexType&gt;
+
+For this example, if we only need the `key_field` and `field2` when specifying
+our `ComplexType`s (because the other fields are optional and we are fine with
+their default values) then we can abbreviate.
+
+Option three generic example:
+```
+some_cwl_field:
+  a_complex_type1: foo   # we accept the default value for field3
+  a_complex_type2: foo2  # we accept the default value for field3
+  a_complex_type3: {}    # we accept the default values for "field2" and "field3"
+```
+
+Option three specific example using [Workflow](Workflow.html#Workflow).[inputs](Workflow.html#WorkflowInputParameter):
+> array&lt;InputParameter&gt; |
+> map&lt;`id`, `type` | InputParameter&gt;
+
+
+```
+inputs:
+  workflow_input01: string
+  workflow_input02: File  # we accept the default of no File format
+```
+
+Option three specific example using [SoftwareRequirement](#SoftwareRequirement).[packages](#SoftwarePackage):
+> array&lt;SoftwarePackage&gt; |  
+> map&lt;`package`, `specs` | SoftwarePackage&gt;
+
+
+```
+hints:
+  SoftwareRequirement:
+    packages:
+      sourmash: [ https://doi.org/10.21105/joss.00027 ]
+      python: {}
+```
+
+
+What if some entries we want to mix the option 2 and 3? You can!
+
+Mixed option 2 and 3 generic example:
+```
+some_cwl_field:
+  my_complex_type1: foo   # we accept the default value for field3
+  my_complex_type2:
+    field2: foo2
+    field3: bar2          # we did not accept the default value for field3
+                          # so we had to use the slightly expanded syntax
+  my_complex_type3: {}    # as before, we accept the default values for both
+                          # "field2" and "field3"
+```
+
+Mixed option 2 and 3 specific example using [Workflow](Workflow.html#Workflow).[inputs](Workflow.html#WorkflowInputParameter):
+> array&lt;InputParameter&gt; |
+> map&lt;`id`, `type` | InputParameter&gt;
+
+
+```
+inputs:
+  workflow_input01: string
+  workflow_input02:     # we use the longer way
+    type: File          # because we want to specify the "format" too
+    format: http://edamontology.org/format_2572
+  workflow_input03: {}  # back to the short form as this entry
+                        # uses the default of no "type" just like the prior
+                        # examples
+```
+
+Mixed option 2 and 3 specific example using [SoftwareRequirement](#SoftwareRequirement).[packages](#SoftwarePackage):
+> array&lt;SoftwarePackage&gt; |  
+> map&lt;`package`, `specs` | SoftwarePackage&gt;
+
+
+```
+hints:
+  SoftwareRequirement:
+    packages:
+      sourmash: [ https://doi.org/10.21105/joss.00027 ]
+      screed:
+        specs: [ https://github.com/dib-lab/screed ]
+        version: [ "1.0" ]
+      python: {}
+```
+
+Note: The `map<…>` (compact) versions are optional, the verbose option #1 is
+always allowed, but for presentation reasons option 3 and 2 may be preferred
+by human readers.
+
+The normative explanation for these variations, aimed at implementors, is in the
+[Schema Salad specification](SchemaSalad.html#Identifier_maps).
+
 ## Identifiers
 
 If an object contains an `id` field, that is used to uniquely identify the