[DNM/investigation only] Pin sphinx in an attempt to fix build documentation fail

[valeriupredoi]

Description

Nightly docs build tests are failing - in all honesty I am not 100% what the cause of that is, so first port of entry is to pin to latest Sphinx (for some reason the env reverts back to 8.1.3 at pip install point - EDIT to be able to get a valid nbsphinx which currently is incompatible with sphinx>=8.2, but that pin is NOT in the conda recipe).

Nbsphinx is a pain in the rear: allowing for Sphinx=8.2.3 solves the build issue no problem https://app.circleci.com/pipelines/github/ESMValGroup/ESMValCore/12855/workflows/d9af7d3a-b4fb-44a4-bc40-c20cee158740/jobs/53453?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link&utm_content=summary

EDIT: this seems to be a transitory issue (with fail frequency rather high)@ same run but failed https://app.circleci.com/pipelines/github/ESMValGroup/ESMValCore/12856/workflows/acec1635-9bcf-43da-9d98-d955cf1d25dd/jobs/53458?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link&utm_content=summary

Gonna have to wait 24h see what happens tomorrow.

Link to documentation:

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🛠 This pull request has a descriptive title and labels
• 🛠 Changes are backward compatible
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.10%. Comparing base (7a1ffd4) to head (bba7b07).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2706   +/-   ##
=======================================
  Coverage   95.10%   95.10%           
=======================================
  Files         257      257           
  Lines       15021    15021           
=======================================
  Hits        14286    14286           
  Misses        735      735           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[valeriupredoi]

this may not be needed after all, it appears to be an internal issue with Sphinx (and manifesting itself for both 8.1 and 8.2) see sphinx-doc/sphinx#13485

[valeriupredoi]

OK nbsphinx is NOT the issue here

[valeriupredoi]

just to point out the flakiness of this whole spiel: de6f818 passed all the tests!

[valeriupredoi]

OK iris is indeed causing the issue: just with iris in the env:

WARNING: Failed to import esmvalcore.iris_helpers.
Possible hints:
* Error: unsupported locale setting
* AttributeError: module 'esmvalcore' has no attribute 'iris_helpers'
WARNING: Failed to import esmvalcore.dataset.
Possible hints:
* AttributeError: partially initialized module 'iris' from '/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/iris/__init__.py' has no attribute '_constraints' (most likely due to a circular import)

• iris==3.10 is FINE
• iris==3.11 is NOT fine
• cf-units is NOT the problem, pinned at 3.3.0 and all passes fine with iris=3.10

The issue is only iris>=3.11
Now, to dig into what's causing it from iris

Another hint, as Bouwe has already found out, iris=3.11 is causing the issue only with Python==3.13.3 (as iris3.10 is compatible only up to and including Python 3.13.2); also, local (ie on my laptop) builds go through fine with iris=3.11 and python=3.13.3.

This is the doc build failed with NO cf-units installed by us (though it is in the env from a dep that needs it), but with iris=3.11 and Python 3.13.3 https://app.circleci.com/pipelines/github/ESMValGroup/ESMValCore/12990/workflows/07fd887b-217b-403d-a9ef-7b18152b21cf/jobs/53941?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link&utm_content=summary - pure issue with locale

Going down the 🐰 hole, this is the only change in Python 3.13.3 (see log https://docs.python.org/3/whatsnew/changelog.html) that deals with locale: python/cpython#129646 specifically python/cpython#129647

FYI @bouweandela

[valeriupredoi]

locale at start:

LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=C.UTF-8

locale after install, before running sphinx:

LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=C.UTF-8

locale.getlocale() and locale.getencoding() return ('en_US', 'UTF-8') and UTF-8 when stuff's breaking

the same things are ('en_US', 'UTF-8') UTF-8 when things are not breaking (on RTD), so this thing is most probably not due to any locale issue, or maybe there could be issues with LC_NUMERIC that I have yet to test ->

Scratch that! On CircleCI it will pass only with:

locale.getlocale() = ('C', 'UTF-8') as it comes out of Python 3.13.2 so there is some sanity here!

LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=C.UTF-8

[bouweandela]

#2708 has been merged