Skip to content

Unit tests are very very slow #207

@brews

Description

@brews

Unit tests run in CI have become very very slow. The can take about 15 minutes to complete.

I suspect dodola.services.validate() using dask.delayed is the culprit. Dask was used here as a hack to work around memory problems (see #126). The drawback to this hack has been that simple unit tests on Datasets covering a long time range take much more time to run.

If this is the case I see two options:

  1. Re-write validation to work around the original memory problems in dodola validate-dataset reads in entire zarr store? #126 while also making unit tests complete faster. i.e. replace the hack solution with a "real" solution.
  2. Find and refactor the one or two long-running unit tests so that they test the same behavior, but run faster.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions