Pin to "distributed >= 2025.5.1" dependency #876
Description
In reviewing the PR #875 with the new required Dask version (2025.5.1) I noticed a recurring error (see below) relating to the distributed package of Dask at the version I had by default and upon investigation it turns out Dask specifies a minimum version of distributed (a closely-related library to Dask they have apparently recently started 'version-locking' with it) but only as an optional dependency rather than an enforced one (see https://github.com/dask/dask/blob/c37e61378ecb15a92963d8936a4a45f89fb98f3d/pyproject.toml#L58), hence it not being upgraded when installing the latest cf tools.
When I update the distributed version to the minimum set in the optional dependency these recurring errors all go away. So, clearly too early a version of distributed causes problems and therefore it seems safest to provide a standard/compulsory dependency version pin in cf-python to ensure dask and distributed play nice. @davidhassell let me know if you agree, then I can add one here. (Note: I am yet to check if the cfdm test suite runs fine without the optional dependency of distributed, but for safety I advise we also pin it there).
Errors repeating with distributed earlier than the Dask optional dependency
Using Python 3.12,main of cfdm and cf-python, from running the cf-python test suite or e.g. python test_regrid.py individually, with distributed-2024.8.2:
======================================================================
ERROR: test_Field_regridc_3d_field (__main__.RegridTest.test_Field_regridc_3d_field)
3-d Cartesian regridding with Field destination grid.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/slb93/git-repos/cf-python/cf/test/test_regrid.py", line 590, in test_Field_regridc_3d_field
x = src.regridc(
^^^^^^^^^^^^
File "/home/slb93/git-repos/cf-python/cf/decorators.py", line 71, in precede_with_kwarg_deprecation_check
operation_method_result = operation_method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/cfdm/decorators.py", line 44, in inplace_wrapper
processed_copy = operation_method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/slb93/git-repos/cf-python/cf/field.py", line 13585, in regridc
return regrid(
^^^^^^^
File "/home/slb93/git-repos/cf-python/cf/regrid/regrid.py", line 705, in regrid
regridded_data = src.data._regrid(
^^^^^^^^^^^^^^^^^
File "/home/slb93/git-repos/cf-python/cf/data/data.py", line 2386, in _regrid
dx = dx.map_blocks(
^^^^^^^^^^^^^^
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/dask/array/core.py", line 2749, in map_blocks
return map_blocks(func, self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/dask/array/core.py", line 905, in map_blocks
out = blockwise(
^^^^^^^^^^
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/dask/array/blockwise.py", line 241, in blockwise
v, collections = unpack_collections(v)
^^^^^^^^^^^^^^^^^^^^^
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/dask/delayed.py", line 182, in unpack_collections
dsk = finalized.__dask_graph__()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/dask/_expr.py", line 1391, in __dask_graph__
from distributed.shuffle._core import P2PBarrierTask
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/distributed/__init__.py", line 23, in <module>
from distributed.actor import Actor, ActorFuture, BaseActorFuture
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/distributed/actor.py", line 13, in <module>
from distributed.client import Future
File "/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/distributed/client.py", line 47, in <module>
from dask.base import collections_to_dsk, tokenize
ImportError: cannot import name 'collections_to_dsk' from 'dask.base' (/home/slb93/miniconda3/envs/cf-env-312-numpy2/lib/python3.12/site-packages/dask/base.py)
----------------------------------------------------------------------
Ran 14 tests in 7.134s
FAILED (errors=3)