Support for ExpressionTool in convert

[simleo]

In convert, We are currently bailing out when an ExpressionTool is encountered:

if hasattr(cwl_tool, "expression"):
    raise RuntimeError("ExpressionTool not supported yet")

Can we support the conversion of ExpressionTool?

If the above clause is removed and we let the processing continue, it crashes because the plan for the activity corresponding to the execution of the ExpressionTool is not found. More specifically, resolve_plan returns None and the program crashes when it tries to do:

plan_tag = plan.id.localpart

Error message:

AttributeError: 'NoneType' object has no attribute 'id'

Adding some prints in resolve_plan:

    def _resolve_plan(self, activity):
        print("resolving plan for", activity.id)
        job_qname = activity.plan()
        print("  job qname:", job_qname)
        plan = activity.provenance.entity(job_qname)
        print("  plan:", plan)
        if not plan:
            m = SCATTER_JOB_PATTERN.match(str(job_qname))
            if m:
                plan = activity.provenance.entity(m.groups()[0])
        return plan

We get:

resolving plan for id:a9f719bd-9bf2-42a4-aa4a-163eb95351dd
  job qname: wf:main/
Entity wf:main/ not found in Provenance<urn:uuid:4b66a4db-eb94-43fe-8475-14d38ac3a3bc from /home/simleo/work/wf_run_crate/expression_tool/cwl/ngtax-run-1/metadata/provenance/primary.cwlprov.xml>
  plan: None

So the activity.plan() (job_qname) is just wf:main/, with no tool-specific tag after the slash. Compare this to the output for a tool in the conversion of tests/data/revsort-run-1:

resolving plan for id:f81dd60b-46db-4e58-b9f9-5606de1f10de
  job qname: wf:main/rev
  plan: entity(wf:main/rev, [prov:type='prov:Plan', prov:type='wfdesc:Process'])

Looking at primary.cwlprov.json:

  "wasAssociatedWith": {
    ...
    "_:id11": {
      "prov:activity": "id:a9f719bd-9bf2-42a4-aa4a-163eb95351dd",
      "prov:agent": "id:ed5680f3-84f4-423c-be6f-d9ed9991a436",
      "prov:plan": "wf:main/"
    },
    ...
}

prov:plan is also wf:main/ for other ExpressionTools used in the workflow. So this is something that's not supported by CWLProv.

The above results have been obtained by trying to convert the RO of an execution of https://gitlab.com/m-unlock/cwl/-/raw/main/workflows/workflow_ngtax.cwl with https://gitlab.com/m-unlock/cwl/-/raw/main/tests/ngtax/ngtax.yaml. The version of cwltool used was 3.1.20240112164112.

[stain]

@mr-c is this missing from cwltool export to CWLProv?