Skip to content

Commit 6c7385c

Browse files
committed
harmonize references to "CWL document(s)"
and other cleanups
1 parent aac1404 commit 6c7385c

File tree

1 file changed

+7
-6
lines changed

1 file changed

+7
-6
lines changed

CONFORMANCE_TESTS.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ system some needed GNU-like tools like `greadlink`.
105105

106106
## Format of the conformance test file
107107

108-
Conformance tests consist of input CWL descriptions plus an input CWL object
108+
A single conformance test consist of the path to an input CWL document plus an input CWL object
109109
and the expected outputs (or `should_fail: true` if the test is deliberately broken)
110110

111111
They are stored in [`conformance_tests.yaml`](https://github.com/common-workflow-language/cwl-v1.2/blob/main/conformance_tests.yaml)
@@ -128,12 +128,13 @@ We will use this single entry to explain the format
128128
```
129129
- `doc`: A sentence that explain what is being tested. Will be printed at test execution time. Should be unique.
130130
- `label`: a short list of underscore (`_`) separated words that succinctly identifies and explains the test.
131-
- `tool` the path to the CWL description to run
131+
- `tool` the path to the CWL document to run
132132
- `job`: the CWL input object in YAML/JSON format. If there are no inputs then use `tests/empty.json`.
133133
- `output` [the CWL output object expected.](#output-matching)
134134
- `tags`: a yaml list of tag names, see [the list of canonical tags below](#tags-for-conformance-tests).
135135

136-
Because this is a `schema-salad` processed document, `$import` can be used to organize the tests into separate files.
136+
Because `conformance_tests.yaml` is a `schema-salad` processed document, [`$import`](https://www.commonwl.org/v1.2/SchemaSalad.html#Import)
137+
can be used to organize the tests into separate files.
137138

138139
Currently, the main file is too big (over 3400 lines); we are slowly re-organizing it.
139140

@@ -151,7 +152,7 @@ In each test entry there is an `output` field that contains a mapping of the exp
151152
If a particular value could vary and it doesn't matter to the proper functioning of the test, then it can be represented by the special token `Any`.
152153

153154
At any level, if there is an extra field, then that will be considered an error.
154-
An exception to this is `class: File` and `class: Directory` objects, the `cwl-runner` under test can add additional fields here.
155+
An exception to this is `class: File` and `class: Directory` objects, the `cwl-runner` under test can add additional fields here without causing a test to fail.
155156
Likewise, if you don't want to test some aspect of a `class: File` or `class: Directory` object (like `nameext`) you can just omit it.
156157

157158
[According to the CWL standards](https://www.commonwl.org/v1.2/CommandLineTool.html#File), the format of the `location` field in
@@ -166,10 +167,10 @@ Additionally, for `class: Directory` the location reported by the actual executi
166167
## Writing a new conformance test
167168

168169
To add a new conformance test:
169-
1. Ensure the CWL description you have tests the desired feature or aspect.
170+
1. Ensure the CWL document you have tests the desired feature or aspect.
170171
2. Run your test using the CWL reference runner (`cwltool`) or another CWL runner
171172
that shows the correct behavior to collect the output, or confirm that validation/execution fails as expected
172-
3. Add the CWL description and output object to the subdirectory `tests` in this repository.
173+
3. Add the CWL document and output object to the subdirectory `tests` in this repository.
173174
4. Fill out a new entry in [conformance_tests.yaml](conformance_tests.yaml) following the [format of the conformance test file](#format-of-the-conformance-test-file)
174175
5. Send a pull request to [current staging branch for the next revision of the CWL standards](https://github.com/common-workflow-language/cwl-v1.2/tree/1.2.1_proposed)
175176
with your changes

0 commit comments

Comments
 (0)