RFC: Remove custom __deepcopy__ method from DimCoord

[jfrost-mo]

🚀 Pull Request

Description

Currently a request for comment, as there is probably a better way to fix this, and I've not done all the checklist stuff yet.

The custom copy implementation for DimCoords causes a RecursionError under python 3.14 as the super() proxy object is now copied, instead of its copy implementation being used on the original object.

This change does lose the read-only latching on copy that DimCoords currently have, which is what makes me think there is a better way. On the other hand, is the DimCoord not being created with read-only points coordinates?

Fixes #6762

---

Consult Iris pull request check list

---

Add any of the below labels to trigger actions on this PR:

• benchmark_this Request that this pull request be benchmarked to check if it introduces performance shifts

[jfrost-mo]

Unsurprisingly this broke the tests that were testing for it.

FAILED lib/iris/tests/unit/coords/test_DimCoord.py::Test_copy::test_writable_points - Failed: DID NOT RAISE <class 'ValueError'>
FAILED lib/iris/tests/unit/coords/test_DimCoord.py::Test_copy::test_realdata_readonly - Failed: DID NOT RAISE <class 'ValueError'>
==== 2 failed, 10420 passed, 54 skipped, 2403 warnings in 477.34s (0:07:57) ====

[ESadek-MO]

From @SciTools/peloton:

Although this obviously is a problem, we're worried that deleting deepcopy() is far too dangerous a way forward. We don't yet have an alternative solution, theoretically we believe that creating a deepcopy handler that never calls super would be the correct solution.

This would also need to be tested against 3.14 (our CI doesn't cover this currently).

[jfrost-mo]

As a short term workaround, it might be worth rebuilding the conda-forge package so it explicitly doesn't support python 3.14, as currently environments that don't pin the python version are broken.

[trexfeathers]

As a short term workaround, it might be worth rebuilding the conda-forge package so it explicitly doesn't support python 3.14, as currently environments that don't pin the python version are broken.

Why conda-forge but not PyPI?

[jfrost-mo]

Only because I forgot people install things from PyPI.

[trexfeathers]

Only because I forgot people install things from PyPI.

Well unfortunately this makes the situation more complicated, since the only way to update PyPI is via a full-on patch release within the Iris GitHub. Since we offer semantic versioning - critical to supporting operational use cases - this involves careful branch management. Our What's New process is not optimised for such light-footed releases either, so that's something to improve in future. And of course we would potentially need to do it all again once Iris has been made compatible (depending on the situation).

Given the frequency of incompatible changes in our dependencies, this would represent an enormous distraction and a significant threat to productivity. I'm not willing to put the team through that.

[jfrost-mo]

The conda-forge package can be updated in just the feedstock by specifying - python >={{ python_min }},<3.14 as the run dependency, and bumping the build number. This might be enough, as pip won't be managing python versions anyway, so there is less chance of accidentally using an incompatible version of python there.

However, like you say, it might not be worth it, as it would take someone's time, the change would have to be put back once it was fixed, and users can always specify their python version explicitly when creating their environment. At least this issue fails completely, rather than subtly, so users will know there is an issue.

[trexfeathers]

as pip won't be managing python versions anyway

Are you sure? I am pretty sure I have had Pip both install dependencies and catch incompatibilities before. Have I misunderstood?

---

If we adopted a policy of letting PyPI 'go out of date' between minor releases then I agree this becomes trivially easy. I had assumed that the clear answer to this is "No", but the cost-benefit might say otherwise.

[jfrost-mo]

By "pip won't be managing python versions", I mean that you can't install python itself through pip, so you will have had to explicitly install python 3.14 to be affected. Most stable distributions or existing installations will have an older version of python, so won't be affected.

As an excuse to learn the PyPI BigQuery API I've run the numbers:

• The conda-forge package was downloaded 39,898 times for releases after (and including) iris 3.12.1, which came out just under 6 months ago.
• The PyPI package (all versions) has been downloaded 105,616 times over the past 6 months.

Therefore, while these download numbers are only approximately comparable¹, it indicates that most of iris's downloads are via PyPI, and thus that probably sways the cost-benefit analysis towards not changing anything.

Footnotes

1. Both in download number calculation method, but and also in usage. I suspect that pip environments are created more frequently, given they are quicker to make, and that they are used on more ephemeral platforms, such as AWS Lambda, and therefore the number of users is probably less that the number of downloads represents. ↩