Skip to content

Commit 180486e

Browse files
committed
changelog 350f0b...1612321
1 parent 9351742 commit 180486e

File tree

6 files changed

+240
-48
lines changed

6 files changed

+240
-48
lines changed

.github/config/wordlist.txt

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ amstutz
33
arrayschema
44
arvados
55
auditability
6+
autogenerated
67
avro
78
backchannel
89
backrub
@@ -16,6 +17,7 @@ bnf
1617
bogdan
1718
bowtie
1819
changelog
20+
charette
1921
checkpoint
2022
checkpointing
2123
checksums
@@ -41,6 +43,7 @@ commandoutputparameter
4143
commandoutputrecordfield
4244
commandoutputrecordschema
4345
commonwl
46+
conformant
4447
contrib
4548
complextype
4649
coresMin
@@ -57,6 +60,7 @@ cwltype
5760
cwlversion
5861
cyclic
5962
dataflow
63+
deserialize
6064
dev
6165
dirent
6266
docafter
@@ -156,6 +160,7 @@ mebibytes
156160
ménager
157161
metacharacters
158162
metaschema
163+
migneault
159164
misgendering
160165
mrc
161166
multipleinputfeaturerequirement
@@ -168,6 +173,7 @@ nekrutenko
168173
networkaccess
169174
neuro
170175
nolinkcheck
176+
ogc
171177
onlinepubs
172178
opengroup
173179
operationinputparameter
@@ -184,6 +190,7 @@ outputsource
184190
outputparameter
185191
outputrecordfield
186192
outputrecordschema
193+
parsers
187194
pathname
188195
petermac
189196
pickvalue
@@ -267,6 +274,8 @@ transcode
267274
txt
268275
typedef
269276
typedsl
277+
undeploy
278+
unserialized
270279
uc
271280
ucdavis
272281
udocker

CommandLineTool.yml

Lines changed: 103 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -52,39 +52,27 @@ $graph:
5252
See also the [Schema-Salad v1.2.1 changelog](SchemaSalad.html#Changelog_for_v1.2.1)
5353
5454
## Changelog for v1.2.1
55-
56-
* The schema for `Requirement`s has changed to enable faster parsing by
57-
autogenerated libraries. The `class` field is now a static enum with a
58-
single permissible value instead of a generic string (for example:
59-
`class: DockerRequirement` for a `DockerRequirement` hint or requirement.)
60-
This allows for autogenerated CWL parsers to recognize any requirement
61-
immediatelly instead of having to check for matching field names and
62-
valid values, as was done previously.
63-
* Likewise, the schema for `CommandLineTool` has changed to enable faster parsing by
64-
autogenerated libraries. The `class` field is now a static enum with a
65-
single permissible value (`class: CommandLineTool`) instead of a generic
66-
string.
67-
This allows for autogenerated CWL parsers to recognize any `CommandLineTool`
68-
immediatelly instead of having to check for matching field names and
69-
valid values, as was done previously.
70-
* The schema for the `CommandLineBinding.position` field now has an explicit
71-
default value of `0`. Previously this was only expression textually in
72-
the description of that field.
73-
* The schema for the `File.streamable` field now has an explicit default
74-
value of `false` to match the textual description.
75-
76-
Note: Other fields like `ResourceRequirement.coresMin`, `.coresMax`,
77-
`.ramMin`, `.ramMax`, `.tmpdirMin`, `.tmpdirMax` have not had defaults
78-
set in the schema so that parsers can discriminate easily between a
79-
value not provided and the default value (`0`).
80-
* Everywhere the schema allows a value of type `long` we also explictly
81-
allow a value of type `int`. By JSON rules this is implicit, but by
82-
making it explicit we aid autogenerated CWL libraries especially in
83-
languages such as Java.
55+
* CWL has been assigned an official IANA Media Type of [`application/cwl`](https://www.iana.org/assignments/media-types/application/cwl)
56+
for either JSON or YAML format. For JSON formatted CWL documents, [`application/cwl+json`](https://www.iana.org/assignments/media-types/application/cwl+json)
57+
has also been assigned and can be used. For specifying a YAML formatted
58+
CWL document, one can use `application/cwl+yaml` but that is not an official
59+
IANA media-type yet; as of 2023-07-23 the `+yaml` suffix has yet to be approved.
60+
The above has been documented in the [Syntax](#Syntax) section.
61+
* There is now an unofficial [JSON Schema for CWL documents](https://github.com/common-workflow-language/cwl-v1.2/blob/1.2.1_proposed/json-schema/cwl.yaml),
62+
donated by Francis Charette-Migneault. This schema captures much, but not
63+
all, of the potential complexity of CWL documents. It was created for
64+
the [draft](https://docs.ogc.org/DRAFTS/20-044.html)
65+
[OGC API - Processes - Part 2: Deploy, Replace, Undeploy](http://www.opengis.net/doc/IS/ogcapi-processes-2/1.0)
66+
standard.
67+
To support the testing of this unofficial JSON Schema for CWL, some of
68+
the `should_fail: true` tests have had the label `json_schema_invalid`
69+
added.
70+
* For consistency, all references to `URI`s have been replaced with `IRI`s
71+
(Internationalized Resource Identifiers).
8472
* The [difference between `$()` and `${}`](#Expressions_(Optional)) were
8573
clarified. We now make more explicit that `${...}` evaluates to
8674
`(function() { ... })()`.
87-
* The publisher of this document is now explictly named; it is the
75+
* The publisher of this document is now explicitly named; it is the
8876
[Common Workflow Language project](https://www.commonwl.org), a member
8977
project of [Software Freedom Conservancy](https://sfconservancy.org/).
9078
* The [Parameter References](#Parameter_references) section has been updated
@@ -100,9 +88,70 @@ $graph:
10088
in the newly added [glossary](#Glossary): they are nonsensical values
10189
that are swapped out with a real value later in the evaluation process.
10290
Workflow and tool expressions **should not** rely on it nor try to parse it.
91+
* The purpose and valid circumstances for using `CommandLineTool.id` has
92+
been made more explicit: It is a unique identifier for that CommandLineTool;
93+
Only useful for `CommandLineTool`s in a `$graph`. This `id` value should
94+
not be exposed to users in graphical or terminal user interfaces.
95+
* How `cwl.output.json` is used to perform [output binding](#Output_binding)
96+
has been clarified, especially with regard to any `path` and `location`
97+
fields for `File` and `Directory` objects referenced within. Two new
98+
conformance tests (`json_output_path_relative`, `json_output_location_relative`)
99+
have been added to verify these clarifications.
100+
* The [BNF grammar description of CWL Parameter References](#Parameter_references)
101+
has been reformatted so that symbols get `code formatting`. `::` is
102+
replaced with `::=` (meaning that the symbol on the left must be
103+
replaced with the expression on the right).
104+
* The example expansion of the [`stdin` shortcut](#stdin) erroneously used
105+
`${ }` instead of `$( )`; this has been corrected.
106+
107+
### Clarifications to the schema in CWL v1.2.1 to aid autogenerated libraries
108+
Many different CWL parsers are autogenerated from the official CWL schema
109+
by using [`schema-salad --codegen`](https://schema-salad.readthedocs.io/en/latest/#codegen-examples).
110+
111+
In CWL v1.2.1 we made many clarifications to the schema to enable faster
112+
parsing; or to produce better results for end users. These changes do not change
113+
the CWL syntax or its meaning; we are just now modeling it better.
114+
115+
* The schema for `Requirement`s has changed to enable faster parsing by
116+
autogenerated libraries. The `class` field is now a static enum with a
117+
single permissible value instead of a generic string (for example:
118+
`class: DockerRequirement` for a `DockerRequirement` hint or requirement.)
119+
This allows for autogenerated CWL parsers to recognize any requirement
120+
immediately instead of having to check for matching field names and
121+
valid values, as was done previously.
122+
* Likewise, the schema for `CommandLineTool`; the `class` field is now a
123+
static enum with a single permissible value (`class: CommandLineTool`)
124+
instead of a generic string.
125+
* The schema for the `CommandLineBinding.position` field now has an explicit
126+
default value of `0`. Previously this was only expression textually in
127+
the description of that field.
128+
* The schema for the `File.streamable` field now has an explicit default
129+
value of `false` to match the textual description.
130+
131+
Note: Other fields like `ResourceRequirement.coresMin`, `.coresMax`,
132+
`.ramMin`, `.ramMax`, `.tmpdirMin`, `.tmpdirMax` have not had defaults
133+
set in the schema so that parsers can discriminate easily between a
134+
value not provided and the default value (`0`).
135+
* Everywhere the schema allows a value of type `long` we also explicitly
136+
allow a value of type `int`: [`File.size`](#File), [`ResourceRequirement`](#ResourceRequirement)`.coresMin`,
137+
`ResourceRequirement.coresMax`, `ResourceRequirement.ramMin`, `ResourceRequirement.ramMax`,
138+
`ResourceRequirement.tmpdirMin`, `ResourceRequirement.tmpdirMax`, `ResourceRequirement.outdirMin`,
139+
`ResourceRequirement.outdirMax`, and [`ToolTimeLimit.timelimt`](#ToolTimeLimit).
140+
141+
By JSON rules this is implicit, but by making it explicit we aid
142+
autogenerated CWL libraries especially in languages such as Java.
143+
* The schema for the `default` field of [CommandInputParameter](#CommandInputParameter)
144+
has been expanded from `Any?` to `["null", File, Directory, Any]`
145+
so that autogenerated CWL libraries will deserialize any 'File' or 'Directory'
146+
objects automatically for the user.
147+
* The schema for the `hints` field of `CommandLineTool` has been expanded
148+
from: `Any[]?` to `["null", { type: array, items: [ ProcessRequirement, Any] } ]`.
149+
This allows autogenerated CWL parsers to deserialize any of the standard
150+
CWL hints instead of forcing the users of those parsers to convert the
151+
unserialized hints to normal objects themselves.
103152
104153
### Updated Conformance Tests for v1.2.1
105-
* Conformance tests are now refered to by their textual identifiers (`id`). Previously
154+
* Conformance tests are now referred to by their textual identifiers (`id`). Previously
106155
this was the `label` field. Tests without a `label`/`id` have been give
107156
one.
108157
* `tests/loadContents/cwloutput-nolimit.cwl`: Made explicit that
@@ -119,15 +168,10 @@ $graph:
119168
a new input parameter, however it does demonstrate the ability to rename
120169
a file and have it staged as a secondary file without having to use
121170
`InitialWorkDirRequirement`.
122-
* `tests/conditionals/cond-wf-002.cwl`, `tests/conditionals/cond-wf-002_nojs.cwl`,
123-
`tests/conditionals/cond-wf-002_nojs.cwl`, `tests/conditionals/cond-wf-011_nojs.cwl`:
124-
Marked the outputs as optional to remove ambiguity for these conditional
125-
`when` tests; allowing conformant CWL runners to be more strict in their
126-
interpretation of the typing rules, if they choose so.
127171
* `directory_input_docker` was incorrectly marked as required, it is optional
128172
unless the feature `ShellCommandRequirement` is stated as being supported.
129173
* `glob_outside_outputs_fails` was incorrectly marked as required, it is
130-
optional unlewss the feature `DockerRequirement` is stated as being supported.
174+
optional unless the feature `DockerRequirement` is stated as being supported.
131175
* `stage_file_array` was incorrectly marked as required, it is optional
132176
unless both the features `InlineJavascriptRequirement` and `DockerRequirement`
133177
are stated as being supported.
@@ -138,6 +182,8 @@ $graph:
138182
* `optional_numerical_output_returns_0_not_null` was incorrectly marked as
139183
required, it is optional unless `InlineJavascriptRequirement` is stated
140184
as being supported.
185+
* `cwloutput_nolimit` and `loadcontents_limit` were incorrectly marked as
186+
optional; this has been corrected as both are required.
141187
* Made it explicit that if a `CommandLineTool` contains logically chained
142188
commands (e.g. `echo a && echo b`) then the `stdout` File/object must
143189
include the output of every command. The `stdout_chained_commands` mandatory
@@ -172,6 +218,19 @@ $graph:
172218
(`inputBinding` for all levels).
173219
* `json_output_path_relative`: Test support for reading cwl.output.json
174220
where 'path' is relative path in output dir.
221+
* `json_output_location_relative`: Test support for reading cwl.output.json
222+
where 'location' is relative reference to output dir.
223+
* `filename_with_hash_mark`: Test for correct handling of URL-quoting in
224+
a filename to refer to filename containing a hash mark.
225+
* `capture_files`: Test that a type error is raised if directories are
226+
returned by `glob` evaluation when type is `File`.
227+
* `capture_dirs`: Test that a type error is raised if files are returned
228+
by `glob` evaluation when type is `Directory`.
229+
* `capture_files_and_dirs`: Test that both files and directories are
230+
captured by `glob` evaluation when type is `[Directory, File]`.
231+
* `very_big_and_very_floats_nojs`: Confirm that very big and very small
232+
numbers are represented on the command line using decimals, not
233+
scientific notation.
175234

176235
### New Optional Conformance Tests for v1.2.1
177236

@@ -181,11 +240,18 @@ $graph:
181240
* `staging-basename`: Use of an `ExpressionTool` to change `basename` of
182241
file, then correctly staging the file using the new name.
183242
* `js-input-record`: A test case for JavaScript with an input record.
243+
* `very_big_and_very_floats`: Confirm that very big and very small
244+
numbers are represented on the command line using decimals, not
245+
scientific notation.
184246

185247
#### `MultipleInputFeatureRequirement`
186248
* `multiple-input-feature-requirement`: MultipleInputFeatureRequirement on
187249
workflow outputs.
188250

251+
#### `InitialWorkDirRequirement`
252+
* `iwd-subdir`: Test emitting a subdirectory from the initial working
253+
directory.
254+
189255
## Introduction to the CWL Command Line Tool standard v1.2
190256

191257
Since the v1.1 release, v1.2 introduces the

0 commit comments

Comments
 (0)