You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- `tags`: a yaml list of tag names, see [the list of canonical tags below](#tags-for-conformance-tests).
135
135
136
-
Because this is a `schema-salad` processed document, `$import` can be used to organize the tests into separate files.
136
+
Because `conformance_tests.yaml` is a `schema-salad` processed document, [`$import`](https://www.commonwl.org/v1.2/SchemaSalad.html#Import)
137
+
can be used to organize the tests into separate files.
137
138
138
139
Currently, the main file is too big (over 3400 lines); we are slowly re-organizing it.
139
140
@@ -151,7 +152,7 @@ In each test entry there is an `output` field that contains a mapping of the exp
151
152
If a particular value could vary and it doesn't matter to the proper functioning of the test, then it can be represented by the special token `Any`.
152
153
153
154
At any level, if there is an extra field, then that will be considered an error.
154
-
An exception to this is `class: File` and `class: Directory` objects, the `cwl-runner` under test can add additional fields here.
155
+
An exception to this is `class: File` and `class: Directory` objects, the `cwl-runner` under test can add additional fields here without causing a test to fail.
155
156
Likewise, if you don't want to test some aspect of a `class: File` or `class: Directory` object (like `nameext`) you can just omit it.
156
157
157
158
[According to the CWL standards](https://www.commonwl.org/v1.2/CommandLineTool.html#File), the format of the `location` field in
@@ -166,10 +167,10 @@ Additionally, for `class: Directory` the location reported by the actual executi
166
167
## Writing a new conformance test
167
168
168
169
To add a new conformance test:
169
-
1. Ensure the CWL description you have tests the desired feature or aspect.
170
+
1. Ensure the CWL document you have tests the desired feature or aspect.
170
171
2. Run your test using the CWL reference runner (`cwltool`) or another CWL runner
171
172
that shows the correct behavior to collect the output, or confirm that validation/execution fails as expected
172
-
3. Add the CWL description and output object to the subdirectory `tests` in this repository.
173
+
3. Add the CWL document and output object to the subdirectory `tests` in this repository.
173
174
4. Fill out a new entry in [conformance_tests.yaml](conformance_tests.yaml) following the [format of the conformance test file](#format-of-the-conformance-test-file)
174
175
5. Send a pull request to [current staging branch for the next revision of the CWL standards](https://github.com/common-workflow-language/cwl-v1.2/tree/1.2.1_proposed)
0 commit comments