[Elephant] Tests and coverage

[psobolewskiPhD]

Tests are currently... incomplete.
We should consider at some point building out some tests to actually test functionality.
Might be able to port some from ezomero which sets up omero in CI for testing:
https://github.com/TheJacksonLaboratory/ezomero/tree/main/tests

Meanwhile, coverage reports flag all CI runs red.
We should consider disabling coverage checks until there is sufficient testing infrastructure.

[will-moore]

We use https://github.com/ome/omero-test-infra on all our OMERO projects.

E.g. used by OMERO.figure https://github.com/ome/omero-figure/blob/master/.github/workflows/omero_plugin.yml

[psobolewskiPhD]

Nice! thanks for the link!
I'm not sure why ezomero has it's own docker compose -- maybe historical.
It does have a lot pre-configured in conf.py so still useful as a reference perhaps.

[jo-mueller]

Will start working on this in the near future. Just to clarify for me: This will restrict tests to Ubuntu, right?

[joshmoore]

omero-test-infra isn't currently multi-platform, i.e. the defaults all use the same base images. But if you're interested in having cross-platform tests we can probably find a way to make that work.

[psobolewskiPhD]

We could have a workflow with a subset of OMERO-interacting tests run on ubuntu.
And then some tests run across other platforms that are just unit tests, test the widgets and use some mocks as needed?

[psobolewskiPhD]

It might also be worth heavily borrowing from the ezomero conftest:
https://github.com/TheJacksonLaboratory/ezomero/blob/main/tests/conftest.py

[joshmoore]

It might also be worth heavily borrowing from the ezomero conftest:

💯 Wonder if it's possible to make that re-usable (within copy-n-paste)...

[psobolewskiPhD]

Could make ezomero a test dependency...

[jo-mueller]

We could have a workflow with a subset of OMERO-interacting tests run on ubuntu.
And then some tests run across other platforms that are just unit tests, test the widgets and use some mocks as needed?

@psobolewskiPhD as for this: Would you set it up so that in the ci you'd check for the runner's OS like that?

      - name: Set up OMERO
        if: runner.os == 'Linux'
        ...spin up OMERO

or would you rather add it as a separate CI workflow step alltogether?

[psobolewskiPhD]

@will-moore @joshmoore what's the ${{ secrets.OMERO_TEST_INFRA_REF }} in that action example?

@jo-mueller A separate Action might be easier, because it might be easier to configure to run the hypothetical omero integration tests only on that? These probably wouldn't require any Qt?
We could break out tests that require omero into their own directory (might be simplest) or use a pytest mark.
Then the other/existing tests would test more of the napari plugin functionality, the widgets, etc. and could use Mock if needed for any OMERO stuff.

[joshmoore]

what's the ${{ secrets.OMERO_TEST_INFRA_REF }} in that action example?

That's the branch name. When there's a large scale breaking change that impacts dozens of repositories, we create a fix branch of test-infra, then all repos can go green without needing a PR/merge cycle.

[psobolewskiPhD]

Once testing is up, then we should revisit the code cov:
Setup a codecov.yml:
https://docs.codecov.com/docs/github-3-customizing-codecov
For now it's disabled to prevent the red x: #126