Skip to content

Commit 1f3ef88

Browse files
authored
New conformance tests must specify a container (#133)
1 parent 4bd3310 commit 1f3ef88

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

CONFORMANCE_TESTS.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -184,7 +184,13 @@ Likewise, please do not test the `path` for `class: File` and `class: Directory`
184184

185185
To add a new conformance test:
186186
1. Ensure the CWL document you have tests the desired feature or aspect.
187-
2. Run your test using the CWL reference runner (`cwltool`) or another CWL runner
187+
2. All `CommandLineTool`s need a software container (via `DockerRequirement`) for better reproducibility, preferrably under `hints`.
188+
Please limit your container usage to the following:
189+
- `dockerPull: docker.io/alpine:latest`
190+
- `dockerPull: docker.io/bash:4.4`
191+
- `dockerPull: docker.io/debian:stable-slim`
192+
- `dockerPull: docker.io/python:3-slim`
193+
4. Run your test using the CWL reference runner (`cwltool`) or another CWL runner
188194
that shows the correct behavior to collect the output, or confirm that validation/execution fails as expected
189195
3. Add the CWL document and output object to the subdirectory `tests` in this repository.
190196
4. Fill out a new entry in [conformance_tests.yaml](conformance_tests.yaml) following the [format of the conformance test file](#format-of-the-conformance-test-file)

0 commit comments

Comments
 (0)