Contribute community partner specific test summaries

A partner may generate test summaries.

In the partner section of collection_rdf_template.yaml we need to specify your partner_id (e.g. 'ilastik') and your workflow which creates and deploys your test summaries.

These dedicated partner test workflows are triggered by the collection CI: trigger_partner_wf.yaml.

The partner repository needs to implement a GitHub Actions workflow which is callable from another workflow and accepts a pending_matrix input:

on:
  workflow_dispatch:
    inputs:
      pending_matrix:
        description: 'json encoded gh matrix of pending validations for new or updated resources (default: test on all resources)'
        required: true
        default: '{"include": [{"resource_id": "**", "version_id": "**"}]}'
        type: string

The workflow should generate test summaries for the resource versions given by pending_matrix. The resource versions can be found on the gh-pages branch of the collection-bioimage-io repository as https://github.com/bioimage-io/collection-bioimage-io/tree/gh-pages/rdfs/<resource_id>/<version_id>/rdf.yaml

Test summaries are mappings with the following keys saved as .yaml files:

bioimageio_spec_version: # str
bioimageio_core_version: # str
error: # Union[None, str, Dict[str, Any]]
name: # str
nested_errors: # Optional[Dict[str, dict]]
source_name: # str
status: # Union[Literal["passed", "failed"], str]
traceback: # Optional[List[str]]
warnings: # dict

The current keys are defined here

• example ilastik:

  • repository with ilastik's test ci: https://github.com/ilastik/bioimage-io-resources
  • this github actions workflow is called by the collection CI: test_bioimageio_resources.yaml
  • deployed test summaries: https://github.com/ilastik/bioimage-io-resources/tree/gh-pages
  • config of this process in collection repo: collection_rdf_template.yaml#L63-L68

• (unfinished) example deepimagej:

  • gh workflow called by collection CI: test_bioimageio_resources.yaml
  • config in collection repo: collection_rdf_template.yaml#L41-L50
  • Specific material from deepImageJ's model repository to run the CI:
    • A drafted GitHub actions workflow to download the installation of Fiji/ImageJ and call the deepImageJ macro that processes the test image and generates an output for each of the models in the repo. The workflow will test that all the models produce the corresponding output tiff file.
    • The workflow calls deepImageJ in headless mode