-
Notifications
You must be signed in to change notification settings - Fork 9
Open
Labels
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is needed
Description
Unit tests run in CI have become very very slow. The can take about 15 minutes to complete.
I suspect dodola.services.validate() using dask.delayed is the culprit. Dask was used here as a hack to work around memory problems (see #126). The drawback to this hack has been that simple unit tests on Datasets covering a long time range take much more time to run.
If this is the case I see two options:
- Re-write validation to work around the original memory problems in dodola validate-dataset reads in entire zarr store? #126 while also making unit tests complete faster. i.e. replace the hack solution with a "real" solution.
- Find and refactor the one or two long-running unit tests so that they test the same behavior, but run faster.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is needed