test case for hash marks in file names (#159)

* add --relax-path-checks to cwltool
* finish label→id migration

Co-authored-by: Michael R. Crusoe <[email protected]>

Author: tetron

.github/workflows/cwltool.yml

@@ -26,4 +26,4 @@ jobs:
         run: pip install cwltest cwltool
 
       - name: Run tests against the reference runner
-        run: ./run_test.sh RUNNER=cwltool EXTRA=--parallel -j$(nproc)
+        run: ./run_test.sh RUNNER=cwltool "EXTRA=--parallel --relax-path-checks" -j$(nproc)

CONFORMANCE_TESTS.md

@@ -57,15 +57,16 @@ Can be mixed with the other test selectors: `-n5-7,15 -N6` == only runs the 5th,
 
 `-s{test_names}`
 
-Run the specific tests according to their `label`s. `{test_names}` is a comma separated list of
-test labels. `-scl_optional_bindings_provided,stdout_redirect_docker,expression_any_null`
+Run the specific tests according to their `id`s. `{test_names}` is a comma separated list of
+test identifiers (the `id` field). `-scl_optional_bindings_provided,stdout_redirect_docker,expression_any_null`
 achieves the same effect as `-n5-7,15 -N6` and it will still work if the tests are re-ordered.
 
 `-S{test_names}`
 
-Excludes specific tests according to their `label`s. `{test_names}` is a comma separated list of
-test labels. `--tags shell_command -Sstderr_redirect_shortcut` will run all test with `expression_tool`
-in their `tags` list except the test with the label `stderr_redirect_shortcut`.
+Excludes specific tests according to their `id`s. `{test_names}` is a comma separated list of
+test identifiers (the `id` field). `--tags shell_command -Sstderr_redirect_shortcut`
+will run all tests with `expression_tool` in their `tags` list except the test
+with the `id` of `stderr_redirect_shortcut`.
 
 ### Misc
 
@@ -125,7 +126,7 @@ The conformance test file is a YAML document: a list of key-value pairs.
 We will use this single entry to explain the format
 ``` yaml
 - doc: Test command line with optional input (missing)
-   label: cl_optional_inputs_missing
+   id: cl_optional_inputs_missing
    tool: tests/cat1-testcli.cwl  
    job: tests/cat-job.json
    output:
@@ -135,7 +136,7 @@ We will use this single entry to explain the format
 - `doc`: A unique, single-line sentence that explain what is being tested.
      Will be printed at test execution time, so please don't make it too long!
      Additional documentation can go as comments in the CWL document itself.
-- `label`: a short list of  underscore (`_`) separated words that succinctly identifies and explains the test.
+- `id`: a short list of  underscore (`_`) separated words that succinctly identifies and explains the test.
 - `tool` the path to the CWL document to run
 - `job`: the CWL input object in YAML/JSON format. If there are no inputs then use `tests/empty.json`.
 - `output` [the CWL output object expected.](#output-matching)
@@ -155,10 +156,6 @@ Example: [`- $import: tests/string-interpolation/test-index.yaml`](https://githu
 adds all the entries in [`tests/string-interpolation/test-index.yaml`](https://github.com/common-workflow-language/cwl-v1.2/blob/main/tests/string-interpolation/test-index.yaml)
 as entries in the main conformance test file.
 
-You may also see references to an `id` field. Don't add them for new tests,
-[we are migrating off of the `id` field](common-workflow-language/cwltest#110) to
-rely only on the `label`s.
-
 ## Output matching
 
 In each test entry there is an `output` field that contains a mapping of the expected outputs names and their values.

conformance_tests.yaml

@@ -1,7 +1,6 @@
-# every "id" must be unique
-# every "label" must be unique and have no spaces
+# every "id" must be unique and have no spaces
 # every "doc" must be unique
-# "label" and "doc" should be meaningful
+# "id" and "doc" should be meaningful
 
 - job: tests/bwa-mem-job.json
   tool: tests/bwa-mem-tool.cwl
@@ -3320,11 +3319,11 @@
   job: tests/record-order-job.json
   output:
     args: [-a, -b, '1', -c, '3', -d, -e, '2', -f, '4']
-  label: record-order-with-input-bindings
+  id: record_order_with_input_bindings
   doc: Test sorting arguments at each level (inputBinding for all levels)
   tags: [ command_line, required ]
 
-- label: output_reference_workflow_input
+- id: output_reference_workflow_input
   output: {
     "last": "me"
   }
@@ -3333,7 +3332,7 @@
     Direct use of Workflow level input fields in the outputs.
   tags: [ required, workflow ]
 
-- label: stdout_chained_commands
+- id: stdout_chained_commands
   output: {
     "out": "a\nb\n"
   }
@@ -3345,7 +3344,7 @@
     and capture the output correctly.
   tags: [ shell_command, command_line_tool ]
 
-- label: multiple-input-feature-requirement
+- id: multiple-input-feature-requirement
   output: {
     "hello_world_in_two_lines": [
       "hello\n",
@@ -3357,7 +3356,7 @@
     MultipleInputFeatureRequirement on workflow outputs.
   tags: [ workflow, multiple_input ]
 
-- label: js-input-record
+- id: js-input-record
   # The output does not have the last \n due to the -n passed to echo
   output: {
     "out": "JS\nwith\nrecord"
@@ -3373,8 +3372,23 @@
   output: {
     "out": "1970-01-01T00:00:00Z\n"
   }
-  label: schemadef_types_with_import
+  id: schemadef_types_with_import
   doc: >-
     Test SchemaDefRequirement with a workflow, with the `$import` under types.
     It is similar to schemadef-wf, but the `$import` is different.
   tags: [ workflow, schema_def ]
+
+- id: filename_with_hash_mark
+  tool: tests/cat-tool.cwl
+  job: tests/octo.yml
+  doc: Test for correct handling of URL-quoting in filename to refer to filename containing a hash mark
+  output: {
+    "output": {
+        "basename": "output",
+        "checksum": "sha1$06b0c59808c236447d065db8f7d2a60de0a805bf",
+        "class": "File",
+        "location": "output",
+        "size": 8
+    }
+  }
+  tags: [ required, command_line_tool ]

tests/octo.yml

@@ -0,0 +1,7 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+file1:
+  class: File
+  location: "octothorpe/item %231.txt"

tests/octothorpe/item #1.txt

@@ -0,0 +1 @@
+item #1

tests/scatter/test-index.yaml

@@ -1,6 +1,5 @@
 - job: scatter-job.yml
-  id: scatter-1
-  label: simple-simple-scatter
+  id: simple_simple_scatter
   tool: simple-simple-scatter.cwl
   output:
     "result": [
@@ -13,8 +12,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter2-job.yml
-  id: scatter-2
-  label: dotproduct-simple-scatter
+  id: dotproduct_simple_scatter
   tool: dotproduct-simple-scatter.cwl
   output:
     "result": [
@@ -27,8 +25,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter3-job.yml
-  id: scatter-3
-  label: simple-dotproduct-scatter
+  id: simple_dotproduct_scatter
   tool: simple-dotproduct-scatter.cwl
   output:
     "result": [
@@ -41,8 +38,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter4-job.yml
-  id: scatter-4
-  label: dotproduct-dotproduct-scatter
+  id: dotproduct_dotproduct_scatter
   tool: dotproduct-dotproduct-scatter.cwl
   output:
     "result": [
@@ -55,8 +51,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter2-job.yml
-  id: scatter-5
-  label: flat-crossproduct-simple-scatter
+  id: flat_crossproduct_simple_scatter
   tool: flat-crossproduct-simple-scatter.cwl
   output:
     "result": [
@@ -81,8 +76,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter3-job.yml
-  id: scatter-6
-  label: simple-flat-crossproduct-scatter
+  id: simple_flat_crossproduct_scatter
   tool: simple-flat-crossproduct-scatter.cwl
   output:
     "result": [
@@ -95,8 +89,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter4-job.yml
-  id: scatter-7
-  label: flat-crossproduct-flat-crossproduct-scatter
+  id: flat_crossproduct_flat_crossproduct_scatter
   tool: flat-crossproduct-flat-crossproduct-scatter.cwl
   output:
     "result": [
@@ -121,8 +114,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter2-job.yml
-  id: scatter-8
-  label: nested-crossproduct-simple-scatter
+  id: nested_crossproduct_simple_scatter
   tool: nested-crossproduct-simple-scatter.cwl
   output:
     "result": [
@@ -155,8 +147,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter3-job.yml
-  id: scatter-9
-  label: simple-nested-crossproduct-scatter
+  id: simple_nested_crossproduct_scatter
   tool: simple-nested-crossproduct-scatter.cwl
   output:
     "result": [
@@ -189,8 +180,7 @@
   tags: [ workflow, subworkflow, scatter, inline_javascript, expression_tool ]
 
 - job: scatter4-job.yml
-  id: scatter-10
-  label: nested-crossproduct-nested-crossproduct-scatter
+  id: nested_crossproduct_nested_crossproduct_scatter
   tool: nested-crossproduct-nested-crossproduct-scatter.cwl
   output:
     "result": [